h'hh 

cTxTlP 


Evaluation of the Texas 
Technology Immersion Pilot 


Outcomes for the Third Year (2006-07) 
January 2008 


Prepared for 

Texas Education Agency 

Prepared by 

Texas Center for Educational Research 



SATCER 

Texas Center for Educational Research 


Tfexas Center for Educational Research 
P.O. Box 679002 
Austin, Ttexas 78767-9002 
800.580.TCER (8237) 

512.467.3632 512.467.3658 (fax) 
tcer.org 


aiTCER 

Texas Center for Educational Research 


Evaluation of the Texas 
Technology Immersion Pilot 
Outcomes for the Third Y ear (2006-07) 


January 2008 


Prepared for 
Texas Education Agency 

Prepared by 

Texas Center for Educational Research 


©Texas Center for Educational Research 




Credits 


Texas Center for Educational Contributing Authors 

Research 


The Texas Center for Educational Research (TCER) 
conducts and communicates nonpartisan research on 
education issues to serve as an independent resource for 
those who make, influence, or implement education policy 
in Texas. A 15-member board of trustees governs the 
research center, including appointments from the Texas 
Association of School Boards, Texas Association of 
School Administrators, and State Board of Education. 

For additional information about TCER research, please 
contact: 

Catherine Maloney, Director 
Texas Center for Educational Research 
12007 Research Blvd. 

P.O. Box 679002 

Austin, Texas 78767-9002 

Phone: 512-467-3632 or 800-580-8237 

Fax: 512-467-3658 


Shapley Research Associates, LLC 
Kelly Shapley, Ph.D. 

Texas Center for Educational Research 
Daniel Sheehan, Ed.D. 

Catherine Maloney, Ph.D. 

Fanny Caranikas- Walker, Ph.D. 


Prepared for 

Texas Education Agency 
1701 N. Congress Avenue 
Austin, Texas 78701-1494 
Phone: 512-463-9734 


Reports are available at www.tcer.org and www.etxtip.info 


Research Funded by 

U.S. Department of Education 



Table of Contents 


Executive Summary i 

Technology Immersion i 

Methodology i 

Evaluation Design i 

Setting and Participants ii 

Data Collection and Analysis ii 

Study Limitations iii 

Major Findings iii 

1. Introduction 1 

Theory of Technology Immersion 1 

Puipose of Study 3 

Theoretical Framework for Technology Immersion 3 

Organization of the Report 5 

2. Methodology 7 

Evaluation Design 7 

Treatment Sample 7 

Control Sample 7 

Characteristics of Participating Schools 8 

Participants 9 

Students 9 

Teachers 10 

Data Collection 11 

Measures 11 

3. Technology Immersion — Third-Year - Implementation 19 

Defining Technology Immersion 19 

Measuring Implementation Fidelity 20 

Implementation Indicators 21 

Computing Implementation Scores 22 

Implementation of Technology Immersion 22 

Implementation Standards 23 

Implementation Indices 31 

Conclusions 32 

4. Effects of Technology Immersion on Teachers and Teaching 35 

Teacher Mediating Variables — HLM Analysis 35 

Level 1: Repeated-Measures Model 36 

Level 2: Teacher-Level Model 36 

Level 3: School-Level Model 37 

Effects of Immersion on Teachers 37 

Technology Knowledge and Skills 38 

Ideology 40 

Student Classroom Activities and Teacher Collaboration 41 

Effects of Immersion on Classroom Practice 41 

Conclusions 44 



5. Effects of Technology Immersion on Students and Learning 47 

Immersion Effects on Student Mediating Variables 47 

HLM Analyses 48 

Immersion Effects on Technology Experiences and Self-Perceptions 49 

Immersion Effects on Student Engagement 57 

Conclusions 61 

6. Effects of Technology Immersion on Student Achievement 63 

Texas Assessment of Knowledge and Skills 63 

Passing Standards and Scale Scores 63 

Standard Scores 64 

Progress in Meeting TAKS Standards 64 

TAKS Reading 64 

TAKS Mathematics 65 

TAKS Social Studies, Science, and Writing 65 

Effects of Immersion on Academic Achievement 67 

TAKS Reading 67 

TAKS Mathematics 71 

TAKS Social Studies and Science 74 

TAKS Writing 76 

Conclusions 76 

7. Factors Associated with Implementation and Outcomes 79 

Factors Associated with Model Implementation 80 

Immersion Components 80 

School Char acteristics 81 

Classroom Immersion 81 

Student Access and Use 82 

Factors Associated with Student Outcomes 83 

TAKS Reading 83 

TAKS Mathematics 85 

Conclusions 87 

8. Conclusions and Implications 89 

Study Limitations 89 

Major Third-Year Findings 90 

Effects of Technology Immersion on Teachers and Teaching 90 

Effects of Technology Immersion on Students and Learning 91 

Effects of Technology Immersion on Academic Achievement 93 

Nature of Third- Year Implementation 96 

References 99 

Appendices 107 



Executive Summary 


The Technology Immersion Pilot (TIP), created by the Texas Legislature in 2003, set forth a vision for 
technology immersion in public schools. Senate Bill 396 called for the Texas Education Agency 
(TEA) to establish a pilot project to “immerse” schools in technology by providing a wireless mobile 
computing device for each teacher and student, technology-based learning resources, training for 
teachers to integrate technology into the classroom, and support for effective technology use. In 
response to this non-funded legislative mandate, the TEA has used more than $20 million in federal 
Title II, Part D monies to fund technology immersion projects for high-need middle schools through a 
competitive grant process. Concurrently, a research study, partially funded by a federal Evaluating 
State Educational Technology Programs grant, is evaluating whether student achievement improves 
over time as a result of exposure to technology immersion. The Texas Center for Educational Research 
(TCER) — a non-profit research organization in Austin — is the TEA’s primary partner for this four- 
year evaluation that began in the 2004-05 school year and will continue through 2007-08. 

Technology Immersion 

State statute provided a general description of technology immersion, but the concept and its 
component parts were defined operationally to foster uniformity. As a way to ensure consistent 
interpretation of the technology immersion model and comparability of implementation across 
schools, the TEA issued a Request for Qualifications (RFQ) that allowed commercial vendors to apply 
to become providers of technology immersion packages. Vendors had to include six components in 
their plan: (a) a wireless mobile computing device for each educator and student on an immersed 
campus to ensure on-demand technology access; (b) productivity, communication, and presentation 
software for use as learning tools; (c) online instructional resources that support the state curriculum in 
English language aits, mathematics, science, and social studies; (d) online assessment tools to 
diagnose students’ strengths and weaknesses or to assess mastery of the core curriculum; 

(e) professional development for teachers to help them integrate technology into teaching, learning, 
and the curriculum; and (f) initial and ongoing technical support. 

Through an expert review process, the TEA selected three lead vendors as providers of technology 
immersion packages (Dell Computer Inc., Apple Computer Inc., and Region 1 Education Service 
Center [ESC]). Package costs, which ranged from about $1,100 to $1,600 per student, varied 
according to the numbers of students and teachers, the type of laptop computer, and the vendor 
provider. Of the 2 1 immersion sites studied in the third year, 5 middle schools selected the Apple 
package, 15 selected the Dell package, and 1 school selected the Region 1 ESC package (Dell 
computer). 


Methodology 


Evaluation Design 

The overarching purpose of the study is to scientifically investigate the effectiveness of technology 
immersion in increasing middle school students’ achievement in core academic subjects as measured 
by the Texas Assessment of Knowledge and Skills (TAKS). The evaluation also examines the 
relationships that exist among contextual conditions, technology immersion, intervening factors 
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(school, teacher, and student), and student achievement. The research design is quasi-experimental 
with middle schools assigned to either treatment or control groups. This report concentrates on 
information gathered during the 2006-07 school year, but analyses also include data from the first 
(2004-05) and second (2005-06) project years. Researchers answered the following questions: 

• How is technology immersion implemented, 

• What is the effect of technology immersion on teachers and teaching, 

• What is the effect of technology immersion on students and learning, 

• Does technology immersion affect student achievement, and 

• What factors are associated with implementation and student outcomes? 

The Theoretical Framework for Technology Immersion guides the evaluation. The experimental 
research design allows an estimate of the effects of the intervention, which is the difference between 
the treatment and control groups. The framework postulates a linear sequence of causal relationships. 
First, experimental schools arc to be “immersed” in technology through the introduction of technology 
immersion components. An improved school environment for technology should then lead to teachers 
who have greater technology proficiency, use technology more often for their own professional 
productivity, collaborate more with their peers, have students use technology more in their classrooms, 
and use laptops and digital resources to increase the intellectual challenge of lessons. In turn, these 
improved school and classroom conditions should lead students to greater technology proficiency, 
more opportunities for peer collaboration, greater personal self-direction, more rigorous and authentic 
learning experiences, and stronger engagement in school and learning. Student mediating variables 
presumably contribute to increased academic performance as measured by standardized test scores. In 
the framework, prior student achievement and student, family, and school characteristics exert their 
own influence on learning. 

Setting and Participants 

The research includes 42 grades 6 to 8 middle schools drawn from rural, suburban, and urban locations 
in Texas. Schools are divided equally between the treatment group (21) and control group (21). The 
middle schools are typically small (402 students, on average); however, enrollments vary widely (from 
83 to 1,447 students). While schools are mainly concentrated in small or very small Texas districts 
(less than 3,000 students), about a third of schools are in very large districts (10,000 or more students). 

The study focused on three student cohorts in the third year. Cohort 1 included eighth graders (2,586 
treatment, 2,863 control) who completed their third project year, Cohort 2 included seventh graders 
(2,644 treatment, 2,882 control) who finished their second project year, and Cohort 3 included sixth 
graders (2,597 treatment, 2,840 control) who concluded their first year. Students in the cohorts were 
predominantly minority (65%) and economically disadvantaged (67%). In the third year, a total of 
1,253 teachers participated in the study, including 591 in immersion schools and 662 in control 
schools. 

Data Collection and Analysis 

Data collection involved a mix of qualitative and quantitative data sources. Researchers conducted site 
visits at each of the middle schools in fall 2004 and again in spring 2005, 2006, and 2007. For this 
report, we concentrate on data gathered through observations in a sample of grades 6, 7, and 8 
classrooms (English language arts, mathematics, social studies, and science). Additional measures 
include annual online teacher surveys and student paper-and-pencil surveys. We also have gathered 
school and student data on a yearly basis from the Texas Public Education Information Management 
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System (PEIMS) and the Academic Excellence Indicator System (AEIS), as well as data on student 
disciplinary actions from schools. 


We used either two- or three-level hierarchical linear models (HLM) to analyze immersion effects on 
teachers’ and students’ perceptions of technology and proficiencies, immersion effects on students’ 
TAKS achievement, and associations between implementation and outcomes. Three-level HLM 
growth modeling estimated the effects of immersion on rates of growth for dependent variables across 
time (2004, 2005, 2006, and 2007). When only two data points were available, we used two-level 
HLM models to estimate the effects of immersion on 2007 scores. For two-level HLM models, we 
calculated effect sizes (ES) in standard deviation units (usually Cohen’s d). Effect sizes greater than 
0.5 are typically interpreted as large, 0.5 to 0.3 as moderate, 0.3 to 0.1 as small, and less than 0.1 as 
trivial. 

Study Limitations 

The sample selection process and matching procedures used with the quasi-experimental design 
appeal - to have produced a sample of schools with good internal validity, in that there are no large, 
statistically significant treatment-control group differences. However, a threat to internal validity was 
introduced in the third year when control schools began to plan for technology immersion and most of 
the control teachers received laptops, instructional resources, and more intensive professional 
development. Generalization of findings to a broader population (external validity) is a primary study 
limitation. Compared to Texas middle-school students as a whole, students in the sample schools are 
substantially more Hispanic and less White and African American. Middle schools are also smaller 
than the statewide average, and schools are located either in small or very small districts (64%) or 
large districts (36%), which differs from the statewide distribution of schools. Additionally, for many 
variables, the study relies on self-reported data from surveys of teachers and students — thus, some 
findings on changes in proficiencies and practices reflect respondents’ perceptions. Nonetheless, the 
triangulation of evidence from multiple sources (surveys, classroom observations, state demographic 
and test databases, multiple student cohorts) verifies the robustness of findings. 

Major Findings 

Effects of Technology Immersion on Teachers and Teaching 

In the third year, immersion teachers continued to grow in technology proficiency and in their 
use of technology for professional productivity at significantly faster rates than control teachers. 

Technology immersion has accelerated teachers’ growth in meeting the state’s Technology 
Application Standards. In a self-assessment of their technology proficiency across four time points, 
immersion teachers considered themselves to be increasingly more technology literate than control 
teachers in areas involving technology operations and pedagogical skills. Similarly, teachers in 
immersion schools are using technology significantly more often for management puiposes, such as 
communicating with students through email and websites, administering assessments, and accessing 
model lesson plans. 

Teachers at schools with higher concentrations of student poverty grew in technology 
proficiency at a slower rate. Consistent with previous years, teachers who taught at schools with 
higher levels of student poverty grew in technology proficiency at significantly slower rates than their 
peers in more advantaged schools. Weaker supports for implementation at higher poverty immersion 
schools may at least partially explain teachers’ slower progress. 
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Teachers in immersion schools expressed increasingly stronger ideological associations across 
years with technology integration and learner-centered practices. Although immersion and control 
teachers initially expressed similar views on instructional practices involving technology, immersion 
teachers altered their instructional beliefs at a significantly faster rate. Thus, immersion teachers 
increasingly employed actions supporting curricular and instructional infusion of technology. 
Immersion teachers also expressed increasingly stronger affiliations across years with constructivist or 
learner-centered practices, such as having students establish individual learning goals and emphasizing 
experiential learning. 

Teachers at immersion schools had more collegial interactions on technology-related issues than 
control teachers, and students used technology more often in immersion classrooms. Teachers at 
immersion schools reported increasingly more frequent collaborative interactions with their colleagues 
that supported instructional practices involving technology than control teachers (e.g., developing 
lesson plans or exchanging information about students), and immersion teachers increased the 
frequency of their students’ Classroom Activities involving technology at a more rapid pace. Although 
student activities with technology have steadily increased in immersion classrooms, third-year 
statistics indicted that students still used various technology resources infrequently (i.e., about once or 
twice a month). While the overall level of classroom technology activities remained low, practices 
varied substantially across teachers and core-subject areas. 

Cumulative evidence suggests that laptop computers and digital resources have allowed students 
in technology immersion schools to experience slightly more intellectually demanding work. New 

resources in technology immersion schools and classrooms are expected to promote students’ higher 
level thinking through more challenging and relevant learning activities that support academic 
achievement. Although observations of core-subject classes in spring of 2005, 2006, and 2007 
revealed no statistically significant differences between the overall Intellectual Challenge of 
immersion and control teachers’ instruction, the sizes of effects favoring immersion teachers increased 
across years. In particular, immersion teachers’ lessons compared to control had a greater emphasis on 
Higher Order Thinking over time. Across both immersion and control classrooms, however, the 
intellectual demand of instruction was typically low (mostly below 2 on the 5-point challenge scale). 

Effects of Technology Immersion on Students and Learning 

Technology immersion significantly increased students’ technology proficiency and reduced the 
proficiency gap between economically advantaged and disadvantaged students. Across three 
cohorts, students in technology immersion schools have made greater progress in mastering the Texas 
Technology Applications standards than control students. Technology immersion had a positive and 
enduring effect on the technology proficiencies of Cohort 1 students from lower socioeconomic 
backgrounds. By the end of the third year, economically disadvantaged eighth graders in immersion 
schools were growing in proficiency at a significantly faster yearly rate than their more affluent 
immersion peers and control-group students. For Cohort 2 (seventh graders) and Cohort 3 (sixth 
graders), technology immersion had a significantly positive effect on technology proficiency for both 
economically advantaged and disadvantaged immersion students. 

Technology immersion significantly increased the frequency of students’ classroom technology 
use and their interactions with peers in small-group activities. Across three cohorts, students in 
immersion schools used technology applications significantly more often in their core-subject 
classrooms than control students. Despite significant increases, third-year statistics (similar to 
teachers’ reports) indicated that students used technology resources infrequently in core classrooms 
(about once or twice a month). Students in immersion schools also had more frequent opportunities to 
learn in small groups with their classmates, whereas control students reported less frequent small- 
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group activities as they advanced to higher grade levels. In general, as immersion teachers altered their 
beliefs about instructional practices, they began to configure classroom activities differently. 

Students at immersion schools, compared to control, reported mounting technical problems over 
time when they used computers at school. Cohorts 1 and 2 immersion students reported increasing 
technical problems using computers across years compared to control students, with the growth in 
problems statistically significant for Cohort 1. Cohort 3 students at immersion schools (sixth graders), 
who inherited laptops that had been used by students during two previous school years, also reported 
significantly more technical problems than control students. Although increased problems appeared to 
accompany aging laptops, mean scores in spring 2007 indicated that students, on average, rarely (a 
few times a year) or just sometimes (once or twice a month) had problems using computers at school. 

Technology immersion and control students regarded themselves as similarly self-directed 
learners. Since the independent and self-guided learning afforded through one-to-one technology was 
expected to positively affect students’ personal self-direction, we asked students to complete the Style 
of Learning Inventory as a measure of Self-Directed Learning. Findings in the third year replicated 
first- and second-year results showing there was no significant immersion effect on students’ self- 
direction. In fact, as both immersion and control students in Cohorts 1 and 2 progressed from lower to 
higher grade levels, their responses to statements measuring self-direction revealed significantly 
negative growth trends. Thus, students reported less self -regulated learning behaviors across time. 

Students in immersion schools had significantly fewer disciplinary actions, similar levels of 
school satisfaction, and significantly lower school attendance rates than control-group students. 

One-to-one computing is often credited with increased student engagement as measured by indicators 
such as stronger commitment to academic work, reduced discipline problems, and increased school 
attendance. Flowever, consistent results for three student cohorts involved in our study show that 
immersion students exhibited significantly stronger school engagement through more positive 
behavior, but they did not express greater satisfaction with school, and they attended school less 
regularly than control students. 

• Behavior and discipline. Disciplinary Action Reports for each student during the 2006-07 
school year, si mi lar to the previous two years, showed that immersion students had 
proportionately fewer behavioral and disciplinary problems than their counterparts in control 
schools. Cohorts 1, 2, and 3 immersion students had an average of 0.65, 0.53, and 0.47 
disciplinary actions, respectively, compared to 0.90, 0.86, and 0.75 for control students. 

• School satisfaction. For each of three cohorts, there was no significant difference in the 
school satisfaction expressed by students at immersion and control schools in the third year. 

• School attendance. Contrary to expectations, across three cohorts, students in immersion 
schools had significantly lower school attendance rates than control-group students. For 
example, at the end of eighth grade. Cohort 1 advantaged students in immersion schools had 
an average attendance rate of 96.3% compared to 97.2% for control students. Economically 
disadvantaged immersion students, similarly, had significantly lower attendance rates than 
their control-group counterparts. Surprisingly, as detailed in the section to follow, immersion 
students’ lower average school attendance was not always associated with lower academic 
achievement. This contrasts with other research linking lower school attendance rates with 
lower test scores. 
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Effects of Technology Immersion on Academic Achievement 


For analyses reported below, students’ TAKS scale scores were standardized and then normalized as T 
scores with a mean of 50 and a standard deviation of 10. We used two-level FILM models and three- 
level FILM growth models to estimate the effects of immersion on students’ test scores. Texas students 
complete TAKS tests annually in reading and mathematics, so reported evidence is stronger for those 
subject areas. In contrast, evidence for science, social studies, and writing is limited because students 
complete those assessments periodically. 

Technology immersion had no statistically significant effect on students’ TAKS reading 
achievement. After controlling for student and school poverty, there were no statistically significant 
effects of immersion on the TAKS reading growth rates for either Cohort 1 (eighth graders) or Cohort 
2 (seventh graders). The immersion effects were positive but not by significant margins. Across 
cohorts, economically disadvantaged students grew in reading achievement at significantly faster rates 
than their more affluent peers. For disadvantaged immersion students, positive annual growth rates 
provided a substantial boost in reading achievement over time. For Cohort 3 sixth graders, after 
controls for students’ prior achievement, demographic characteristics, and school poverty, there was 
no statistically significant effect of immersion on students’ 2007 TAKS reading scores. Similar to the 
other cohorts, the immersion effect was positive but not by a significant degree. 

Technology immersion had a statistically significant effect on TAKS mathematics achievement, 
particularly for economically advantaged and higher achieving students. After controlling for 
student and school poverty, technology immersion had a statistically significant effect on students’ 
TAKS mathematics growth rates for Cohorts 1 and 2 students. For Cohort 1, a significant interaction 
effect revealed that economically advantaged students in immersion schools increased their math 
achievement at a significantly faster rate than disadvantaged immersion students, and at a faster rate 
than both economically advantaged and disadvantaged control-group students. For Cohort 2, 
economically advantaged and disadvantaged immersion students had TAKS mathematics growth rates 
that significantly outpaced their control-group counterparts. For Cohort 3 sixth graders, after 
controlling for students’ prior achievement, demographic characteristics, and school poverty, there 
was a statistically significant effect of immersion that acted through students’ pretest scores. Other 
factors being equal, as students’ TAKS pretest scores increased, the achievement gap favoring 
immersion students over control widened for 2007 TAKS mathematics scores. Thus, immersion had a 
stronger and significant effect on math scores for higher achieving sixth graders. 

Students who had greater access to laptops and used laptops for learning to a greater extent, 
especially outside of school, had significantly higher TAKS reading and mathematics scores. We 

used a series of FILM models to investigate the relationships between implementation levels and 
student academic achievement. Specifically, Student Access and Use was an aggregate 
implementation measure of the extent to which a student had access to a laptop throughout the school 
year (number of days), the frequency of technology use for learning in core-subject classes, and the 
extent of laptop use for homework and learning games. FILM results showed that Student Access and 
Use was a statistically significant positive predictor of students’ TAKS reading and mathematics 
achievement for each of the three student cohorts. Of the three elements of Student Access and Use, 
students’ use of their laptops for Home Learning — a measure of the extent to which students used 
laptops outside of school for homework in the four core-subject areas and for learning games — was 
the strongest predictor of both TAKS reading and mathematics achievement. In contrast, we found that 
reading and mathematics teachers’ reported levels of Classroom Immersion were typically 
insignificant predictors of students’ academic achievement. Results highlight the important role that 
individual laptops play in promoting ubiquitous learning and in equalizing the out-of-school learning 
opportunities for students in disadvantaged family and school situations. 
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The effects of technology immersion on reading and mathematics achievement generally became 
stronger over time as teachers and students became more accomplished technology users. The 

immersion effects on reading and mathematics achievement evolved across three project years. In the 
first project year, the immersion effects on TAKS scores were negative. In the second year, immersion 
effects were typically positive, but not by statistically significant margins. In the third year, 
significantly positive immersion effects on TAKS mathematics emerged for each of three student 
cohorts, and links were established between higher levels of student technology use and achievement. 
These findings underscore the importance of longitudinal studies in assessing the impacts of 
educational initiatives on student academic achievement. 

Evidence regarding the effects of technology immersion on students’ TAKS social studies, 
science, and writing achievement is inconclusive. Since TAKS tests for social studies, science, and 
writing are not administered annually, immersion effects for these subject areas cannot be replicated 
across cohorts and years. Accordingly, it is not possible to draw definitive conclusions about the 
effects of technology immersion for these subject areas. Available results typically show no 
statistically significant effects of immersion, with differences between groups favoring immersion 
students for TAKS social studies and control students for TAKS science and writing. 

• Social studies. After controlling for Cohort 1 eighth graders’ reading achievement (7th grade), 
demographic characteristics, and school poverty, there was no statistically significant effect of 
immersion on 2007 TAKS social studies scores. The immersion effect was positive but not by 
a significant degree. 

• Science. After controlling for prior achievement (5th grade science score), demographic 
characteristics, and school poverty, there was no statistically significant effect of immersion 
on Cohort 1 eighth graders’ TAKS science achievement. The immersion effect was negative, 
and a statistically significant interaction showed that as TAKS pretest scores increased, the 
achievement gap favoring control students over immersion widened for 2007 science scores. 
Thus, there was significantly negative effect on TAKS science scores for higher achieving 
eighth graders at immersion schools. 

• Writing. For both Cohorts 1 and 2, after controlling for pretest writing scores (4th grade 
writing), demographic characteristics, and school poverty, there was no statistically significant 
effect of immersion on students’ TAKS writing scores as seventh graders. The immersion 
effect was negative across both cohorts. 

Nature of Third-Year Implementation 

Although the overall level of implementation increased between the second and third project 
years, just a quarter of schools reached substantial levels of technology immersion. Full 
implementation of the technology immersion model requires Leadership, Teacher Support (buy-in), 
Parent and Community Support, Technical Support, and Professional Development. Given adequate 
supports, teachers are expected to reach high levels of Classroom Immersion, and Student Access and 
Use of technology is expected to be robust. Mean immersion standard scores revealed small increases 
between the second and third implementation years for each of the immersion support components as 
well as for teachers’ overall level of Classroom Immersion. In contrast, the level of Student Access 
and Use was stable across years. Although the quality of schools’ implementation improved slightly in 
the third year, we estimate that about a quarter of middle schools (5) achieved substantial immersion , 
while the remaining schools (16) had minimal to partial immersion levels. Nevertheless, third-year 
results show that technology immersion can have positive effects on teachers and students even at 
lower implementation levels. 
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School administrators advanced implementation through their provision of supports for 
teachers’ technology immersion efforts, whereas teachers’ greater support for immersion along 
with technical support elevated Student Access and Use. Teachers’ opinion of the strength of 
administrative leadership for technology at their school was significantly associated with their 
perceived levels of implementation support (i.e., collective support for technology innovation, parent 
and community support, the prevalence of technical support, and the robustness of professional 
development). Additionally, teachers’ overall support for technology innovation and the extent to 
which they believed that the quality of technical support addressed infrastructure and maintenance 
issues causing banders to students’ laptop use, were significantly associated with greater Student 
Access and Use. To reach higher levels of immersion, many schools needed stronger supports for 
implementation in the third year. 

Core-subject teachers at the majority of schools reported only partial levels of Classroom 
Immersion in the third year; teachers at some schools, however, made collective progress in 
creating technology-immersed classrooms. Immersion standard scores for each of five elements of 
Classroom Immersion showed slightly stronger implementation in the third year, with the largest 
increases for teachers’ ideological affiliations with Technology Integration and Learner-Centered 
Instruction, and the smallest change for Student Activities with technology in the classroom. There 
were notable increases in teachers’ use of technology as a communication tool and for the 
enhancement of their own professional productivity. Core teachers (as a whole) at about a fifth of 
schools reached a substantial level of Classroom Immersion. HLM analyses for individual students 
and their teachers showed that reading and mathematics teachers’ reported levels of Classroom 
Immersion, in most cases, were statistically insignificant predictors of students’ TAKS reading and 
mathematics achievement. Measurement issues, within classroom variability, and interdisciplinary 
teacher effects provide potential explanations for the unexpected results. 

Students’ access to and use of laptops for learning within and outside of school generally fell 
short of substantial to full implementation. Students at more than two-thirds of schools had just 
partial levels of Student Access and Use in the third year, whereas students at about a third of schools 
had only minimal access and use. Students’ opportunities to use their laptops both within classrooms 
and outside of school were affected by the number of days that students actually had their laptops. 
Year-to-year comparisons indicated that students’ Laptop Access Days declined between the second 
and third project years. In contrast, students reported small increases in their use of laptops for Core- 
Content Learning and Home Learning. 

Larger schools and schools with a greater proportion of economically disadvantaged students 
had lower levels of implementation. Overall trends showed that schools with larger student 
enrollments tended to have slightly lower implementation levels than schools with fewer students, and 
schools with higher percentages of economically disadvantaged students tended to have lower 
implementation levels. Technical support was a significant problem at larger schools, whereas 
collective teacher support for technology innovation was a significant issue for schools with greater 
proportions of disadvantaged students. Teachers at higher poverty schools also grew in technology 
proficiency at significantly lower rates, and student access to and use of technology decreased as the 
percentages of minority and economically disadvantaged students increased. In contrast, the schools’ 
achievement context (percentage of students passing all TAKS tests), was positively associated with 
nearly all of the implementation indicators. Clearly, if students are to realize the full potential of 
laptops and technology resources, larger schools and schools serving disadvantaged student 
populations must have adequate supports for technology immersion in place to meet the specific needs 
of the school’s teachers, students, and parents prior to implementing an immersion project. 
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1. Introduction 


Historically, the piecemeal way in which most Texas schools introduced technology into the educational 
process has been an obstacle to the effective use of technology for teaching and learning (Texas 
Education Agency, 2006). Recognizing this limitation, the Technology Immersion Pilot (TIP), created by 
the Texas Legislature in 2003, set forth a vision for technology immersion in public schools. Senate Bill 
396 called for the Texas Education Agency (TEA) to establish a pilot project to “immerse” schools in 
technology by providing a wireless mobile computing device for each teacher and student, technology- 
based learning resources, training for teachers to integrate technology into the classroom, and support for 
effective technology use. In response to this non-funded legislative mandate, the TEA has used more than 
$20 million in federal Title II, Part D monies to fund technology immersion projects for high-need middle 
schools through a competitive grant process. Concurrently, a research study, partially funded by a federal 
Evaluating State Educational Technology Programs grant, is evaluating whether student achievement 
improves over time as a result of exposure to technology immersion. The Texas Center for Educational 
Research (TCER) — a non-profit research organization in Austin — is the TEA’s primary partner for this 
four-year evaluation that began in the 2004-05 school year and will continue through 2007-08. 

Theory of Technology Immersion 

The vision for educational technology endorsed by many educators, leaders, and policymakers has shifted 
in recent years from the use of particular technology software products to technology’s incorporation into 
every aspect of the educational environment. Changing views reflect our growing understanding of how 
students learn and how to create environments that enhance teaching and learning. Cognitive science and 
other research reveal that children learn more when they are engaged in meaningful, relevant, and 
intellectually stimulating work (Bransford, Brown, & Cocking, 2003; Newmann, Bryk, & Nagoaka, 

2001). Many believe that technology can support such learning experiences and also enable students to 
develop competencies needed for the 21st century, such as digital literacy, inventive thinking, and 
effective communication (CEO Forum, 2001; Lempke, Couglin, Thadani, & Martin, 2003; Partnership for 
21 st Century Skills, 2006). 

Similarly, Texas recognizes that the state’s long-term success is tied to the preparation of students for the 
digital age. The Texas Long-Range Plan for Technology, 2006-2020, advances the previous state plan for 
the integration of technology within schools across four domains: teaching and learning; educator 
preparation and development; leadership, administration, and instructional support; and infrastructure for 
technology (TEA, 2006). Senate Bill 396 further defined this comprehensive plan as technology 
immersion. Consistent with the overall Texas vision for technology, the long-term aspiration for 
technology immersion is to “prepare each student for success and productivity as a lifelong learner, a 
world-class communicator, a competitive and creative knowledge worker, and an engaged and 
contributing member of an emerging global society” (TEA, 2006, p. viii). 

While state statute provided a general description of technology immersion, school-based implementation 
of the intervention required additional detail. In specifying the critical components of the immersion 
model, TEA staff considered current research on educational technology as well as practical wisdom 
gained through pilot studies and statewide technology initiatives. Technology immersion assumes that 
effective technology use in schools and classrooms requires robust technology access, technical and 
pedagogical support for implementation, professional development for educators in using technology 
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effectively, and readily available curricular and assessment resources that support the state’s foundation 
curriculum (English language aits, mathematics, science, and social studies). 

First, technology use in schools and classrooms requires robust access. Despite school-level 
improvements in the ratio of students to instructional computers in Texas (Education Week, 2007), recent 
survey data show that an average of 2.9 or less classroom computers is insufficient to allow every student 
access (Shapley, Benner, Heikes, & Pieper, 2002; Shapley et al., 2006b). In response to prevailing 
conditions, technology immersion calls for one-to-one student access to computers. The Texas project, in 
contrast to one-to-one laptop initiatives being implemented in other states and school districts (e.g., 

Maine, Michigan, New Hampshire, Vermont, Henrico County in Virginia) adopts a comprehensive 
approach. In particular, technology immersion also assumes that increased access to and use of 
technology in schools requires adequate technical and pedagogical support. Schools must have robust 
electronic networks to support wireless laptops and digital content. Campus-based support is also vital, as 
ample studies show the importance of on-site support personnel who assist teachers in learning to use 
technology, troubleshooting technical problems, and effectively integrating technology into lessons (e.g., 
National Center for Education Statistics [NCES], 2000; Ringstaff & Kelley, 2002; Ronnkvist, Dexter, & 
Anderson, 2000; Shapley et al., 2002). 

In addition, the technology immersion model assumes that teachers must have effective professional 
development. High-quality professional development, as research demonstrates, is of longer duration and 
provides richer learning experiences, more comprehensive investigation of topics, and time for practice 
and experimentation (e.g., Caret, Porter, Desimone, Birman, & Yoon, 2001; Lawless & Pellegrino, 2007; 
Penuel, Fishman, Yamaguchi, & Gallagher, 2007; Smerdon, et al., 2000). Moreover, when a particular 
technology is mastered over time, it is more likely to be incorporated into instruction (Zhao & Frank, 
2003). Teachers also need follow-up support as they acquire and implement new skills in the instructional 
setting (Bradburn & Osborne, 2007; Garet et al., 2001; Nugent & Fox, 2007; Sulla, 1999). Professional 
development should also focus on subject-specific content or specific teaching methods. For technology, 
this means building teachers’ basic technology skills as well as their understanding of curricular 
integration (CEO Forum, 2000, 2001; Denton, Davis, & Strader, 2001; Ringstaff & Kelly, 2002; Web- 
Based Education Commission, 2000). The alignment of professional development activities with 
teachers’ personal goals for learning is also important in advancing teacher change (Garet et al., 2001; 
Penuel et al., 2007). 

Additionally, technology -related professional development should be paid of broader professional growth 
initiatives in schools (Fullan & Hargreaves, 1996; Mann, Shakeshaft, Becker, & Kottkamp, 1999; 
Newmann & Associates, 1996). Professional development activities that include collective participation 
(e.g., whole schools or teachers of the same subjects or grades) are more likely to be coherent with 
teachers’ experiences and needs (Garet et al., 2001). A leadership development component is crucial, 
since research points consistently to the important role of school leaders in successful implementation of 
technology (Bradburn & Osborne, 2007; Johnston & Cooley, 2001; Pitler, 2005). 

Technology immersion also requires curricular and assessment resources that support the state’s 
curriculum. Thus, laptops in immersion schools include software that allows students and educators to use 
wireless laptops as a tool for teaching, learning, communication, and productivity. Digital resources (e.g., 
online, CD-ROMS, stored on local networks) also provide students with a means for more personalized 
learning activities, and interactive technologies allow them to build new knowledge by doing, receiving 
feedback, and refining their understanding. Technologies also help students to acquire more information, 
visualize difficult-to-understand concepts, and advance understanding (Bransford, Brown, & Cocking, 
2003). Online formative assessments enable teachers to diagnose students’ strengths and needs or to 
assess their mastery of curricular standards. 
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Purpose of the Study 


The overarching purpose of the study is to scientifically investigate the effectiveness of technology 
immersion in increasing middle school students’ achievement in core academic subjects as measured by 
the Texas Assessment of Knowledge and Skills (TAKS). The evaluation also examines the relationships 
that exist among contextual conditions, technology immersion, intervening factors (school, teacher, and 
student), and student achievement. The research design is quasi-experimental with middle schools 
assigned to either treatment or control groups. This report concentrates on information gathered during 
the 2006-07 school year at 21 treatment and 21 control schools. Researchers answered the following 
questions: 

• How is technology immersion implemented, 

• What is the effect of technology immersion on teachers and teaching, 

• What is the effect of technology immersion on students and learning, 

• Does technology immersion affect student achievement, and 

• What factors arc associated with implementation and student outcomes? 

Theoretical Framework for Technology Immersion 

The Theoretical Framework for Technology Immersion guides the evaluation (see Figure 1.1). The 
experimental design, as illustrated in the framework, allows researchers to estimate of the effects of 
technology immersion, which is the difference between the treatment and control groups. We also 
postulate a linear sequence of causal relationships. Program implementation comes first. Experimental 
schools arc to be “immersed” in technology through the introduction of technology immersion 
components. The quality of implementation reflects the robustness of wireless laptop access for teachers 
and students, the adequacy of technical and pedagogical support services to maintain an immersed 
campus, the extent to which professional development supports curricular integration of technology, and 
how well curricular resources and assessments are used. 

Given quality implementation, we expect that an improved school environment for technology should 
then lead to teachers who have greater technology proficiency, have students use technology more and in 
new ways in their classrooms, and use laptops and digital resources to increase the intellectual challenge 
of lessons. In turn, these improved school and classroom conditions should lead students to greater 
technology proficiency, more frequent classroom technology activities, more opportunities for peer 
collaboration, greater personal self-direction, and stronger engagement in school and learning. Student 
mediating variables presumably will contribute to increased academic performance as measured by 
standardized test scores. In the framework, links also are shown between student achievement and 
student, family, and school characteristics, which exert their own influence on learning. The research 
literature underpinning the framework is included in Appendix A. 
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Organization of the Report 


Data collection in the third project year (2006-07) involved a mix of qualitative and quantitative data 
sources. Researchers conducted site visits to each of the middle schools in fall 2004 and again in spring of 
2005, 2006, and 2007. For this report, we concentrate on data gathered through observations in a sample 
of grades 6, 7, and 8 classrooms. Additional measures include annual online teacher surveys and student 
paper-and-pencil surveys. We also have gathered school and student data on a yearly basis from the Texas 
Public Education Information Management System (PEIMS), the Academic Excellence Indicator System 
(AEIS), as well as data on student disciplinary actions from schools. 

Report sections are organized around findings relative to the study’s research questions. An overview of 
report chapters is provided below. 

• Chapter 1, Introduction, provides background on the technology immersion project as well as 
the study’s theoretical framework. The chapter also establishes the purpose for the study and the 
research questions addressed. 

• Chapter 2, Methodology, presents information on the evaluation design, characteristics of 
treatment and control schools, study limitations, study participants, data collection methods, and 
data analysis procedures. 

• Chapter 3, Technology Immersion — Third-Year Implementation, describes progress toward 
implementation in the third year and compares second- and third-year implementation. 

• Chapter 4, Effects of Technology Immersion on Teachers and Teaching, presents findings on the 
effects of immersion on teacher variables, including technology knowledge and skills, ideology, 
student classroom activities and peer collaboration, and the intellectual challenge of lessons. 

• Chapter 5, Effects of Technology Immersion on Students and Learning, offers findings on the 
effects of immersion on mediating variables, including students’ experiences with technology; 
their self-perceptions of technology proficiency, self-directed learning, and school satisfaction; 
and their engagement in school and learning. 

• Chapter 6, Effect of Technology Immersion on Student Achievement, presents findings on the 
effects of technology immersion on academic achievement, as measured by TAKS reading, 
mathematics, writing, science, and social studies. 

• Chapter 7, Factors Associated with Implementation and Outcomes, presents results for 
investigations of factors related to the implementation of technology immersion, and the 
associations between implementation and student academic achievement. 

• Chapter 8, Conclusions and Implications. The final section presents the major findings from the 
study and discusses the implications of outcomes. 


5 




2. Methodology 


Evaluation Design 

The evaluation design is quasi-experimental. Interested districts and associated middle schools responded 
to a Request for Application (RFA) offered by the Texas Education Agency (TEA) to become 
Technology Immersion Pilot (TIP) schools. Applicants had to meet eligibility requirements for Title II, 
Part D funds (i.e., high-need due to children from families with incomes below the poverty line, schools 
identified for improvement, or schools with substantial need for technology). Twenty-two technology 
immersion schools, selected through the competitive grant process, were matched by researchers with 22 
control schools on key characteristics, including size, regional location, demographics, and student 
achievement. Two middle schools from one district (one treatment and one control) were removed from 
analyses in the second year due to damage caused by Hurricane Rita. Thus, third-year results are for 21 
treatment and 21 control schools. A re-analysis of baseline data for the new comparison groups revealed 
no statistically significant differences between school and student characteristics. Thus, the study’s 
research design appeal's sound. 

Treatment Sample 

In spring 2004, the TEA released a series of Requests for Applications (RFAs) inviting school districts to 
apply for TIP grants for up to two middle schools. The agency held an external review of proposals, with 
applications scored and rank ordered. Following the external review, researchers and agency staff 
reviewed proposals to ensure that applications met criteria established for technology immersion. Final 
selection of TIP schools involved the consideration of several factors, including proposal ratings, size, 
location, student diversity, and academic achievement. Decisions were influenced by the need for 
geographic distribution and the availability of comparable schools for the control group pool. Schools 
received grants to support the implementation of technology immersion for four school years. 

Control Sample 

The selection of control campuses first involved the generation of a pool of grades 6 to 8 middle schools 
eligible to receive federal funds for participation in the study. As a next step, researchers identified middle 
schools that matched treatment campuses as nearly as possible on factors, including (a) district and 
campus size, (b) regional location, (c) the proportion of economically disadvantaged and minority 
students, (d) percentage of students passing all TAKS tests, and (e) the gaps between the percentage of 
White students and African American and Hispanic students passing TAKS (all tests). Selection involved 
the use of SPSS ® statistical software procedures to establish parameters around each variable of interest 
and the creation of a computer-generated list of “best matches” for each treatment school. The final 
selection involved a review of the matched list by a team of six researchers to identify the optimal control 
school for each treatment school. Additional schools were selected as alternates in the case that a selected 
control site declined the invitation to participate in the study. This selection process yielded 22 control 
group schools including controls for 8 campuses that came from within the same districts as the treatment 
schools and controls for 14 campuses from closely matched single, middle school districts. For the first 
two evaluation years, each control school received $25,000 annually for study participation, with 25% of 
funds earmarked for professional development as required by Title II, Part D guidelines. 
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At the end of the second year, the TEA offered grants to control schools to begin planning for technology 
immersion. Of the 21 control schools included in analyses, 16 (76%) applied for and received TIP start-up 
grants. Grant guidelines allowed control schools to begin planning for technology immersion in the third 
year (2006-07); teachers could also receive laptops and instructional resources, and schools were 
required, as in previous years, to use 25% of funds for professional development. In the fourth year 
(2007-08), schools will provide laptops for their students, although the majority of schools plan to 
concentrate on single grade levels rather than the entire school. Control schools that declined immersion 
grants continued to receive $25,000 annually for study participation. 

Characteristics of Participating Schools 

The third-year study includes 42 grades 6 to 8 middle schools, including 21 treatment and 21 control 
schools drawn from rural, suburban, and urban locations in Texas. Middle schools are typically small, 
with more than three-quarters enrolling 600 students or less. Schools are highly concentrated in small or 
very small districts (2,999 or less students) across the state, but a third of schools arc in large districts 
(10,000 or more students). There are two campus charter schools (one treatment and one control) located 
in a large urban district. (See statistics in Appendix B .) 

Results for Mcsts show that the percentages of economically disadvantaged, minority, English as a second 
language (ESL), and special education students are statistically equivalent across the treatment and 
control schools (Table 2.1). Likewise, results for student enrollment, mobility, and TAKS passing rates 
show no significant differences. Consequently, the treatment and control schools are sufficiently well 
matched on key demographic and academic performance measures. Additionally, both treatment and 
control groups include a comparable range of campus and district enrollments and schools from diverse 
regions. 

Table 2.1. Comparison of Baseline Characteristics: Technology Immersion ( N = 21) and Control 
Schools (A/ =21) 

!" 95% Confidence Interval for Difference I 


Variable 

Condition 

Mean 

SD 

Lower 

Upper 

t (40) 

Enrollment 

Immersion 

374.9 

348.4 

-284.6 

177.5 

-0.47 


Control 

428.5 

391.3 




Economic disadvantage (%) 

Immersion 

70.8 

17.5 

-3.4 

19.4 

1.42 


Control 

62.8 

19.0 




Minority (%) 

Immersion 

68.1 

28.4 

-10.4 

24.7 

0.83 


Control 

60.9 

27.8 




ESL (%) 

Immersion 

13.5 

17.2 

-1.6 

16.0 

1.66 


Control 

6.3 

9.9 




Special education (%) 

Immersion 

14.7 

5.5 

-4.0 

1.8 

-0.76 


Control 

15.8 

3.7 




Student mobility (%) 

Immersion 

15.8 

4.6 

-3.8 

2.8 

-0.30 


Control 

16.3 

5.9 




TAKS 2004, Passing All (%) 

Immersion 

52.4 

15.7 

-9.2 

8.5 

-0.08 


Control 

52.8 

12.5 




TAKS 2003, Passing All (%) 

Immersion 

65.9 

11.4 

-9.1 

5.5 

-0.50 


Control 

67.6 

12.0 





Source: Texas Education Agency AEIS reports 2004 

Note. TAKS = Texas Assessment of Knowledge and Skills. Differences between groups are statistically 
insignificant. Two campuses (one treatment and one control) were excluded from the groups in the second year. 


Considering baseline statistics, the sample selection process and matching procedures appear to have 
produced a sample of schools with good internal validity, in that there are no large, statistically significant 




treatment-control differences. Still, the tendency for immersion schools to enroll somewhat higher 
percentages of minority, economically disadvantaged, and limited English proficient students could affect 
outcomes given known links between disadvantaged status and lower achievement (Sirin, 2005). Another 
threat to internal validity was introduced in the third project year when control schools began to 
implement elements of the treatment. As noted above, control schools began to plan for technology 
immersion in the third year, and most of the control teachers received new laptops and instructional 
resources. And, while teachers at control schools had opportunities for technology-related professional 
development during the first two project years, the emphasis intensified in the third year as schools 
purchased professional development services from vendors (Dell/Pearson and Apple) that focused on 
elements of technology immersion. Introducing portions of technology immersion in control schools 
could underestimate the magnitude of the treatment effect in the third year. 

Another limitation of the study is external validity — the extent to which the results of an experiment can 
be generalized from the specific sample to the general population. Schools eligible to become paid of the 
treatment group were limited to those serving children from families living in poverty 1 and middle 
schools with grades 6 to 8. Only schools that applied for the grant, and submitted applications that met a 
threshold of quality, were eligible for consideration. Due to these restrictions, the treatment group is not 
representative of the average middle school in Texas. 

The majority of students in the sample arc economically disadvantaged, with 67% of sample students 
qualifying for federal free or reduced-price lunch compared to 51% for middle schools statewide. Sample 
schools include substantially more Hispanic and less White and African American students than state 
averages for middle schools. Overall, about 58% of sample students are Hispanic compared to about 37% 
of Texas middle school students. Conversely, the sample includes fewer African American students (7% 
vs. 14%) and White students (36% versus 46%) compared to the state averages. The sample schools also 
differ structurally from Texas middle schools as a whole. Middle schools in Texas, on average, enroll 
more students (667 vs. 402 in sample schools). Sample schools are located either in small or very small 
districts or large districts, whereas state middle schools are distributed across very small or small, mid- 
sized, and large districts. Differences between sample schools and the state almost certainly reflect 
funding restrictions (Title II, Part D) and the amount of available funds per grant. The maximum grant 
amount ($750,000) fell well short of the amount required to support one-to-one technology in larger 
middle schools. 


Participants 


Students 

Four groups or cohorts of students are followed by the study, with Cohort 1 followed for four years, 
Cohort 2 for three years. Cohort 3 for two years, and Cohort 4 for one year (Table 2.2). Data collection in 
2006-07 centered on Cohorts 1, 2, and 3. Cohort 1 (eighth graders) included a total of 5,449 students, with 
2,586 students enrolled at treatment campuses and 2,863 at control campuses; Cohort 2 (seventh graders) 
included 5,526 students, with 2,644 at treatment campuses and 2,882 at control campuses; and Cohort 3 
(sixth graders) included 5,437 students, with 2,597 students at treatment campuses and 2,840 at control 
campuses. 


1 Federal definition used: 27% of population or more than 2,500 people living below poverty line. 
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Table 2.2. Student Cohorts by School Year and Grade 



Middle School 

High School 

Year 

Grade 6 

Grade 7 

Grade 8 

Grade 9 

2004-05 

Cohort 1 


2005-06 

Cohort 2 

Cohort 1 



2006-07 

Cohort 3 

Cohort 2 

Cohort 1 


2007-08 

Cohort 4 

Cohort 3 

Cohort 2 

Cohort 1 


Note. Bold text denotes the current evaluation year. 


Table 2.3 shows that about three-fourths of eighth graders (Cohort 1), seventh graders (Cohort 2), and 
sixth graders (Cohort 3) are economically disadvantaged. Comparison groups have similar proportions of 
disadvantaged and minority students, and female and male students. The main difference between groups 
is the greater proportion of limited English proficient (LEP) students in treatment schools (about 9 to 13 
percent more). Treatment schools also have slightly higher percentages of economically disadvantaged 
and Hispanic students. 

Table 2.3. Demographic Characteristics of Students: 2006-07 

Enroll- Eco Ethnicity 

ment Disadv. AA Hispanic White LEP Gender 

Cohort 1 

Treatment 

N 2,586 1,959 151 1,881 527 587 1,257 1,329 

% 47.5 75.8 5.8 72.7 20.4 22.7 48.6 51.4 

Control 

N 2,863 2,078 213 1,920 713 380 1,385 1,478 

% 52.5 72.6 7.4 67.1 24.9 13.3 48.4 51.6 

Cohort 2 

Treatment 

N 2,644 2,043 150 1,966 496 732 1,312 1,332 

% 47.8 77.3 5.7 74.4 18.8 27.7 49.6 50.4 

Control 

N 2,882 2,106 234 1,963 666 415 1,400 1,482 

% 52.2 73.1 8.1 68.1 23.1 14.4 48.6 51.4 

Cohort 3 

Treatment 

N 2,597 2,050 128 1,961 496 775 1,257 1,340 

% 47.8 78.9 4.9 75.5 19.1 29.8 48.4 51.6 

Control 

N 2,840 2,119 195 1,991 640 605 1,403 1,437 

% 52.2 74.6 6.9 70.1 22,5 21.3 49.4 50.6 

Note. Spring 2007 student database collected from 21 treatment and 21 control schools 


Teachers 

During the 2006-07 school year, 1,253 teachers participated in the study, including 591 at treatment 
campuses and 662 at control campuses (Table 2.4). Teachers in comparison groups arc remarkably similar 
in terms of gender, ethnicity, advanced degrees, and average teaching experience. The number of teachers 
declined in the second and third years due to the exclusion of two campuses. 
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Table 2.4. Demographic Characteristics of Teachers by Year 



2004-05 

2005-06 

2006-07 


Treatment 

Control 

Treatment 

Control 

Treatment 

Control 


N=22 

N=22 

N=21 

N=21 

N=21 

N=2I 

Number of teachers 

622 

682 

604 

653 

591 

662 

% Female 

65.4 

68.8 

63.4 

68.3 

66.5 

69.2 

% Minority 

42.4 

35.3 

44.9 

43.3 

43.1 

42.9 

% African American 

7.8 

7.5 

2.8 

4.8 

3.2 

5.3 

% Hispanic 

32.2 

26.3 

40.4 

37.3 

39.9 

37.6 

% White 

57.6 

64.7 

55.1 

56.7 

55.2 

56.0 

% with no degree 

0.0 

2.0 

0.2 

0.3 

0.2 

1.4 

% with advanced degree 

21.7 

22.2 

21.2 

18.3 

19.3 

18.1 

Average years experience 

10.9 

11.4 

10.6 

11.5 

10.8 

11.4 


Note. The total number of teachers was 1,304 in 2004-05. 1,257 in 2005-06, and 1,253 in 2006-07. 


Data Collection 

Data collection for the project began in August 2004. As Table 2.5 illustrates, researchers conducted site 
visits at each of the middle schools in fall 2004 and in spring of 2005, 2006, and 2007. Additional 
measures, administered as pre-tests in fall and post-tests in spring, included teacher online surveys and 
student paper-and-pencil surveys. Additionally, we gathered school and student demographic, attendance, 
and achievement data from the Texas Public Education Information Management System (PEIMS) and 
Academic Excellence Indicator System (AEIS). In spring 2005, 2006, and 2007, individual middle 
schools submitted student-level data on disciplinary actions. 


Table 2.5. Time Frame for Data Collection by Year 



2004-05 

2005-06 

200< 

3-07 


Fall 

Spring 

Fall 

Spring 

Fall 

Spring 


2004 

2005 

2005 

2006 

2006 

2007 

Site visits (classroom observations ) 

X 

X 


X 


X 

Teacher Questionnaire (all teachers) 

X 

X 


X 


X 

Teacher Questionnaire (new teachers) 



X 


X 


Student Questionnaire and SLI (Cohort 1) 

X 

X 


X 


X 

Student Questionnaire and SLI (Cohort 2) 



X 

X 

X 

X 

Student Questionnaire and SLI (Cohort 3) 





X 

X 

Texas Assessment of Knowledge/Skills (TAKS) 


X 


X 


X 

Attendance 


X 


X 


X 

Disciplinary actions 


X 


X 


X 


Note. Data collection for 22 treatment and 22 control schools in 2004-05 and 21 treatment and 21 control schools in 2005-06 and 
2006-07. TAKS and attendance data were collected for spring 2003 through 2007. SLI = Style of Learning Inventory. 


Measures 

Instruments measuring mediating and outcome variables included surveys and student performance 
measures. Survey items and scale scores reliabilities are provided in Appendix C. 

Teacher Questionnaire 

Immersion and control teachers completed an online technology survey in fall 2004 (September to 
October) and spring 2005 (April to May). Additionally, in fall 2005 and 2006 (September to October), 
teachers new to the schools completed the survey, and in spring 2006 and 2007 (April to May), all 
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teachers completed surveys. The survey included items related to school technology, teachers’ technology 
proficiency and use, and professional development experiences. In fall 2004, 1,271 teachers completed 
surveys (97% of all teachers, 97% of treatment, and 98% of control). In spring 2005, 1,144 teachers (88% 
of all teachers, 87% of treatment, and 88% of control ) completed surveys. In spring 2006, 1,175 teachers 
completed surveys (93% of all teachers, 92% of treatment, and 95% of control). In spring 2007, 1,208 
teachers completed surveys (94% of all teachers, 94% of treatment, and 93% of control). 

School mediating variables. Teachers responded to 33 items pertaining to their perceptions of school 
technology. They rated their strength of agreement with statements on a 5-point scale ranging from 1 
(, strongly disagree ) to 5 ( strongly agree). Maximum likelihood factor analysis with Varimax rotation 
revealed five distinct factors, including Leadership (12 items). Classroom Technology Integration (4 
items), Technical Support (5 items), Innovative Culture (4 items), and Parent and Community Support (2 
items). Measures of internal consistency (Cronbach’s alpha) for school-level factors ranged from 0.66 to 
0.97. 

Teacher mediating variables. Teacher surveys included measures of mediating valuables, with items 
pertaining to teachers’ perceptions of Technology Proficiency (27 items), Professional Productivity (17 
items), Student Classroom Activities (17 items), and Collaboration (1 1 items related to teacher 
interactions with colleagues). Additionally, confirmatory factor analysis of items adapted from the Levels 
of Technology Implementation (LoTi) Questionnaire (Moersch, 2001) showed reasonable fit indices for a 
model having Technology Integration (10 items), Learner-Centered Instruction (4 items), and Resistance 
to Integration (3 items) as factors. Cronbach’s alpha reliability coefficients for scales ranged from 0.70 to 
0.98. 

For Technology Proficiency items, teachers indicated their skill level on a 7-point scale with 1 and 2 
indicating low proficiency ( not true of me now), 3, 4, and 5 indicating moderate proficiency ( somewhat 
true of me now), and 6 and 7 indicating proficiency (very true of me now). Measures of integration — 
Technology Integration, Learner-Centered Instruction, and Resistance to Integration — also involved a 
7-point scale ranging from 1 ( not true of me now) to 7 (very true of me now). For Professional 
Productivity, Student Classroom Activities, and Collaboration, teachers used a 5-point scale to rate the 
frequency of activities or interactions: 1 (never), 2 (rarely- e.g., a few times a year), 3 (sometimes-e.g., 
once or twice a month), 4 (often-e.g., once or twice a week), and 5 (almost daily). 

Student Surveys 

Students completed a paper-and-pencil questionnaire measuring their technology proficiency and use, and 
the Style of Learning Inventory: ( SL1 ), a measure of self-directed learning (i.e., self-generated behaviors 
oriented toward the attainment of learning goals). Cohort 1 students completed surveys as sixth graders in 
fall 2004 and spring 2005, as seventh graders in spring 2006, and as eighth graders in spring 2007. Cohort 
2 students completed surveys as sixth graders in fall 2005 and spring 2006 and as seventh graders in 
spring 2007. Cohort 3 students (sixth graders) completed technology surveys in fall 2006 and spring 
2007. 

Technology survey. Survey items measured students’ Technology Proficiency (22 items), Classroom 
Activities (12 items), Technical Problems (6 items), Small-Group Work (6 items), and School 
Satisfaction (6 items). Cronbach’s alpha coefficients ranged from 0.77 to 0.94. As a measure of 
Technology Proficiency, students indicated how well they could use various technology applications on a 
5-point scale: 1 (I can do this not at all or barely), 2 (I can do this with some difficulty), 3 (1 can do this 
fairly well), 4 (I can do this very well), and 5 (I can do this extremely well). For measures of Classroom 
Activities, Technical Problems, and Small-Group Work, students used a 5-point scale to rate the 
frequency of activities or interactions: 1 (never), 2 (rarely— e.g., a few times a year), 3 (sometimes-e.g., 
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once or twice a month), 4 {often-e. g., once or twice a week), and 5 ( almost daily). Students rated school 
satisfaction items on a 5-point agreement scale ranging from 1 ( strongly disagree ) to 5 ( strongly agree). 

Technology survey response rates for students are summarized in Table 2.6. Response rates were in the 
80% to 9 1 % range from fall 2004 through spring 2007. In each time period, there were only small 
differences in response rates between cohorts and comparison groups. Note that in spring 2007 response 
rates were slightly reduced because surveys were not administered at three campuses, two treatment and 
one control school. 


Table 2.6. Student Technology Survey Response Rates: 2004-05, 2005-06, and 2006-07 



Fall 3 

Spring 2005 

Spring 2006 

Spring 2007 


N 

% 

N 

% 

N 

% 

N 

% 

Cohort 1 

Treatment 

2,319 

90 

2,053 

80 

2,291 

87 

2,168 

84 

Control 

2,505 

84 

2,485 

83 

2,544 

87 

2,473 

86 

All 

4,824 

87 

4,538 

82 

4,835 

87 

4,641 

85 

Cohort 2 

Treatment 

2,209 

84 

— 

— 

2,379 

89 

2,228 

84 

Control 

2,405 

86 

— 

— 

2,452 

87 

2,363 

82 

All 

4,614 

85 

— 

— 

4,831 

88 

4,591 

83 

Cohort 3 

Treatment 

2,233 

86 

- 

- 

- 

- 

2,220 

85 

Control 

2,584 

91 

- 

- 

- 

- 

2,464 

87 

All 

4,817 

89 

- 

- 

- 

- 

4,684 

86 


‘‘Students completed surveys as sixth graders in fall 2004, 2005, and 2006. 


Style of Learning Inventory. The SLI is a 48-item survey, developed by the Metiri Group (2004), that is 
based on a model of self-regulated learning (Schunk & Zimmerman, 1998). The items on the SLI are 
categorized into 12 scales and three groupings. The three grouping and related scales are listed below. 

• Forethought is defined as influential processes and beliefs that precede efforts to learn (goal 
setting, strategic planning; self-efficacy beliefs; goal orientation; and intrinsic interest), 

• Performance/Volition control refers to processes that occur during learning efforts and affect 
concentration and performance (attention focusing, self-instruction, imagery; self-monitoring; and 
help seeking), and 

• Self-reflection involves processes that occur after learning efforts and influence a learner’ s 
reaction to that experience. Since the learning process is cyclical, these processes will in turn 
influence forethought regarding subsequent learning efforts (self evaluation, attributions, self 
reactions, and adaptivity). 

Students rated statements regarding their personal self-direction on a 7-point scale, ranging from 1 
{completely false) to 7 {completely true). Confirmatory factor analysis of fall 2004 SLI data revealed low 
convergent validity of the scales and groupings and no discriminant validity. In addition, the scales and 
groupings were not internally consistent (a = 0.18 to 0.52). Because of these findings, analyses were 
limited to the SLI total score (a = 0.89). 

Table 2.7 summarizes SLI response rates. Response rates ranged from 71% to 89% across time periods. 
With the exception of the spring 2005 SLI administration, there were only small differences in response 
rates between cohorts or comparison groups. 
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Table 2.7. Style of Learning Inventory Response Rates: 2004-05, 2005-06, and 2006-07 



Fall 11 

Spring 2005 

Spring 2006 

Spring 2007 


N 

% 

N 

% 

N 

% 

N 

% 

Cohort 1 1 

Treatment 

2,142 

83 

2,174 

85 

2,116 

80 

2,152 

83 

Control 

2,442 

82 

2,120 

71 

2,387 

81 

2,472 

86 

All 

4,584 

82 

4,294 

77 

4,503 

81 

4,624 

85 

Cohort 2 

Treatment 

2,115 

80 

— 

— 

2,198 

82 

2,201 

83 

Control 

2,265 

81 

— 

— 

2,228 

79 

2,368 

82 

All 

4,380 

80 

— 

— 

4,426 

80 

4,569 

83 

Cohort 3 

Treatment 

2,173 

84 

— 

— 

— 

— 

2,209 

85 

Control 

2,534 

89 

— 

— 

— 

— 

2,434 

86 

All 

4,707 

87 

- 

- 

- 

- 

4,643 

85 


■“Students completed the Style of Learning Inventory as sixth graders in fall 2004, 2005, and 2006. 


Observation of Teaching and Learning 

Researchers have conducted classroom observations for teachers who instructed Cohorts 1, 2, and 3 
students. In fall 2004 and spring 2005, we observed in a sample of sixth-grade, core-subject classrooms 
(reading/English language arts, mathematics, science, and social studies). In spring 2006, we observed a 
sample of classrooms including sixth-grade teachers (Cohort 2 students) and seventh-grade teachers 
(Cohort 1 students). In spring 2007, we observed a sample of classrooms including sixth-grade teachers 
(Cohort 3 students), seventh-grade teachers (Cohort 2 students), and eighth-grade teachers (Cohort 1 
students). 

The Observation of Teaching and Learning (OTL) form documents basic descriptive information (e.g., 
number of students, content area), technology access and use (i.e., technology available and used by the 
teacher and students), and classroom environment (i.e., organization and management). In addition, 
researchers used time-interval ratings to record information in six areas: class organization (e.g., 
individual students, pairs, small groups, whole group), teacher activities (e.g., directing, guiding 
substantive discussion), teacher’s technology use (e.g., peripherals, presentation software), student 
activities (e.g., listening, learning facts, definitions, algorithms), students’ technology use (e.g., express 
themselves in writing, learn/practice skills), and student engagement (rated on a 5-point scale from low 
engagement to high engagement). 

Observers made the first rating after observing for 5 minutes, then made a rating every 10 minutes. 

During the observation, observers also recorded descriptive notes on the lesson objectives, teachers’ 
questioning strategies (lower or higher order), and class activities. Observations lasted about 45 minutes. 
After the observation, and based on time-interval ratings and descriptive notes, observers rated the 
intellectual challenge of classroom work. Relying on rubrics developed by Newmann, Secada, and 
Wehlage (1995), observers rated four standards measuring the intellectual quality of classroom instruction 
on a 5-point scale: Higher Order Thinking, Disciplined Inquiry, Substantive Conversation, and Value 
Beyond School. An aggregate score across three of the standards was used as an overall measure of the 
Intellectual Challenge of instruction. We excluded the Substantive Conversation standard because ratings 
were biased by teachers’ classroom organization. Classes with teacher-directed instruction typically 
provided more public conversations, and thus, better opportunities to document the nature of 
conversational exchanges. 

Number of observations. During fall 2004, researchers conducted observations at half of middle schools 
(11 treatment and 1 1 control). Subsequently, we expanded observations to all of the middle schools. In 
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fall 2004, researchers observed 125 classrooms (60 treatment and 65 control); in spring 2005, we 
conducted follow-up observations, when possible, in the same classrooms. Altogether, we observed 206 
classrooms (105 treatment and 101 control) in spring 2005. The following year (spring 2006), we 
observed 217 classrooms (114 treatment and 103 control). These observations included a nearly equal 
mix of sixth- and seventh-grade classrooms. In spring 2007, we observed 194 classrooms (95 treatment 
and 99 control). These observations included a combination of sixth-, seventh-, and eighth-grade 
classrooms. At small campuses, researchers observed nearly all core-subject teachers. For larger 
campuses, we observed a representative sample of core teachers. 

Training procedures. Prior to site visits in fall 2004 and spring of 2005, 2006, and 2007, researchers 
participated in one- or two-day training events. Training activities informed data collectors about the 
research design, aspects of technology immersion, data collection protocols, effective interview and focus 
group techniques, and classroom observation procedures. Approximately half of each training event was 
devoted to the establishment of inter-rater agreement on the OTL form. During observation training, 
raters first reviewed background information and individual item and code definitions in the OTL manual. 
Raters next viewed a video in which a classroom teacher used technology as paid of a lesson. The trainer 
stopped raters at 10-nrinute intervals to record ratings, discuss the extent of agreement or disagreement, 
and resolve misunderstandings. This process was repeated for an additional classroom video. 

To further enhance inter-rater agreement, raters were paired for observations in classrooms during visits 
to a middle school selected for training puiposes. Following paired classroom observations in these 
schools, raters again discussed assigned ratings and resolved disagreements. Subsequently, for site visits 
to treatment and control middle schools, observers were paired for about 25% of classroom observations. 
Overlapping observations allowed the calculation of the consistency of observers’ scores (i.e., the 
percentage of agreement on ratings from paired observations). Additionally, paired observations 
supported the use of Many-Facet Rasch Measurement (MFRM) to adjust scores on the Intellectual 
Challenge factor for differences across raters. 

Inter-rater agreement. Inter-rater agreement on the rating scales for the Intellectual Challenge standards 
(Higher-Order Thinking, Disciplined Inquiry, Substantive Conversation, and Value Beyond School) was 
established by calculating the percentage of time observers agreed on ratings from paired observations. 
Analyses of observations from fall 2004 indicated 78% inter-rater agreement. Agreement reached 98% 
when scale categories were allowed to vary by one scale point (on the 5-point scale). Inter-rater 
agreement declined somewhat in spring of 2005, 2006, and 2007. Exact agreement was 63%, 62%, and 
62%, respectively, and 89%, 93%, and 96% when ratings varied by one scale point. 

Reliability of scores. Statistics for inter-rater agreement indicated that raters may have had somewhat 
different standards for assigning scores, so we needed to adjust statistically for the differences in the 
severity of raters. An overall measure of Intellectual Challenge for each teacher was constructed using 
MFRM. The quality of instruction measure is an aggregate score across three standards (Higher Order 
Thinking. Disciplined Inquiry, and Value Beyond School). The measure is adjusted for the relative 
difficulty of each standard and the relative severity (or leniency) of each observer. MFRM analysis 
produces several fit statistics that can be used to measure each observer’ s intrarater reliability or internal 
consistency. One of these, observer infit, weights each standardized residual by its variance and is more 
sensitive to unexpected patterns of small residuals. A second statistic, observer outfit, is an unweighted 
mean-square residual sensitive to outlying residuals (Linacre, 2004). 

There is no fixed rule for setting upper and lower limits for theses fit statistics. “Misfitting” raters have 
been defined as having either a mean-square infit or outfit statistic greater than 1.5 (Lunz. Wright, & 
Linacre, 1990), or the range has been from 0.5 to 3.0 (Myford & Wolfe, 2000). We define a “misfitting” 
observer as one with either a mean-square infit or outfit statistic less than 0.5 or greater than 1.5. This 
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defines “misfit” as less than 50% of the variance in ratings than is modeled (a muted pattern) and more 
than 50% of the variance than is modeled (a noisy pattern). Observation data in fall 2004, spring 2005, 
spring 2006, and spring 2007, respectively, resulted in observer infit values from 0.61 to 1.34, 0.61 to 
1.34, 0.43 to 1.59, and 0.58 to 1.14, and observer outfit values from 0.62 to 1.20, 0.62 to 1.20, 0.40 to 
1.67, and 0.66 to 1.17. While the spring 2006 fit statistics extended slightly beyond the 0.5 to 1.5 range, 
mean infit and outfit values were in the 0.90 to 1.00 range. No unusual rating patterns appeared to be 
present in the spring 2006 classroom observation data, with only slightly unpredicted or overly 
predictable ratings (Linacre, 1995). 

Texas Assessment of Knowledge and Skills (TAKS) 

The TAKS is Texas’ criterion-referenced assessment that annually measures students’ mastery of the 
state’s content standards. TAKS assesses reading at grades 3 to 9; English language aits at grades 10 and 
11; writing at grades 4 and 7; mathematics at grades 3 to 11; science at grades 5, 8, 10, and 11; and social 
studies at grades 8, 10, and 11. Stringent quality control measures are applied at all stages of test 
administration, scanning, scoring, and reporting. Internal consistency reliabilities for TAKS assessments 
are in the high .80s to low .90s range. Evidence also supports the content, construct, and criterion-related 
validity of TAKS assessments. 2 

Table 2.8 shows the TAKS completion schedule for Cohorts 1, 2, and 3 students. Students complete 
TAKS reading and mathematics assessments annually, so all student cohorts have pretest and posttest 
measures. Additionally, Cohort 1 students completed TAKS science in 2004 (5th grade) and 2007 (8th 
grade), and TAKS social studies in 2007. Cohort 2 students completed the TAKS writing assessment in 
2004 (4th grade) and 2007 (7th grade). 

Table 2.8. Texas Assessment of Knowledge and Skills Completion Schedule by Student Cohort 

Texas Assessment of Knowledge and Skills (TAKS) 



F 

leading 

Mathematics 

\ 

Yritim 

X 

? 

Social Studies 

< 

Science 

Year 

Cl 

C2 

C3 

Cl 

C2 

C3 

Cl 

C2 

C3 

Cl 

C2 

C3 

Cl 

C2 

C3 

2003 
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- 

- 

X 

- 

- 

X 

- 

- 

- 

- 

- 

- 

- 

- 

2004 

X 

X 

- 

X 

X 

- 

- 

X 

- 

- 

- 

- 

X 

- 

- 

2005 

X 

X 

X 

X 

X 

X 

- 

- 

rx 

- 

- 

- 

- 

X 

- 

2006 

X 

X 

X 

X 

X 

X 

X 

- 

- 

- 

- 

- 

- 

- 

- 

2007 

X 

X 

X 

X 

X 

X 

- 

X 

- 

X 

- 

- 

X 

- 

- 


Note. Cl = Cohort 1, C2 = Cohort 2, and C3 = Cohort 3. Italic text means the TAKS score was used as a pre-test measure. 

At grades 6, 7, and 8, TAKS reading measures four objectives: understanding of culturally diverse written 
texts, knowledge of literary elements, use of strategies to analyze written texts, and application of critical- 
thinking skills. TAKS mathematics at grades 6, 7, and 8 measures six objectives: numbers, operations, 
and quantitative reasoning; patterns, relationships, and algebraic reasoning; geometry and spatial 
reasoning; concepts and uses of measurement; probability and statistics; and mathematical processes and 
tools used in problem solving. 

At grade 7, TAKS writing measures six objectives: given a context, produce an effective composition for 
a specific puipose; demonstrate a command of conventions of spelling, capitalization, punctuation, 
grammar, usage, and sentence structure; recognize appropriate organization of ideas in written text; 
recognize correct and effective sentence construction in written text; recognize standard usage and 
appropriate word choice in written text; proofread for correct punctuation, capitalization, and spelling in a 
written text. At grade 8, TAKS science measures five objectives: nature of science; living systems and the 


Technical information is available on the Texas Education Agency website at http://www.tea.state.tx.us/student 
assessment/resources/techdig04/index.html. 
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environment; structures and properties of matter; motion, forces, and energy, and earth and space systems. 
Grade 8 TAKS social studies measures five objectives: history, geography, economics and social 
influences, political influences, and social studies skills. 

School Attendance and Disciplinary Actions 

Post-measures of student attendance for Cohort 1 came from PEIMS data for the 2004-05, 2005-06, and 
2006-07 school years; attendance data from 2003-04 served as the pre-measure. Similarly, for Cohort 2, 
student attendance data for 2005-06 and 2006-07 provided post-measures while data from 2004-05 served 
as the pre-measure. Likewise, for Cohort 3, student attendance data for 2006-07 provided a post-measure 
and data from 2005-06 served as the pre-measure. Additionally, individual campuses submitted data for 
student disciplinary actions taken during the 2006-07 school year. Data files included an indicator for the 
total number of Disciplinary Action Reports (PEIMS 425 records) reported for each student (Cohorts 1 , 2, 
and 3) during the school year. 
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3. Technology Immersion — Third-Year Implementation 


Researchers have investigated the implementation of technology immersion across three project years. 
Second-year findings showed that many of the 2 1 treatment schools continued to have difficulty 
implementing the prescribed components of the technology immersion model. Implementation supports 
for immersion generally did not meet full implementation standards, and accordingly, overall 
implementation at the classroom and student levels was low. Still, implementation levels varied by 
campus and about a third of schools reached levels that more nearly met substantial to full immersion 
standards. Given known associations between implementation quality and outcomes (e.g., Berman & 
McLaughlin, 1978; Borman, 2005; Borman, Hewes, Overman, & Brown, 2003; Datnow, Borman, & 
Stringfield, 2000), we continued to monitor schools’ progress in the third year. This chapter begins with a 
description of technology immersion and the use of technology immersion packages as a means to 
operationally define the treatment and ensure more consistent implementation across sites. Next, we 
describe our approach to measuring implementation. Finally, findings are presented on the fidelity of 
third-year implementation at the treatment schools, and comparisons are made between implementation 
for the second (2005-06) and third (2006-07) project years. 

Defining Technology Immersion 

To promote consistent interpretation of the technology immersion model and comparability of 
implementation across schools, the Texas Education Agency (TEA) issued a Request for Qualifications 
(RFQ) that allowed commercial vendors to apply to become providers of technology immersion packages 
(TEA, 2003). State statute provided a general description of technology immersion, but the concept and 
its component parts were defined operationally to foster uniformity. Vendors had to include six 
components in their plan: 

• A wireless mobile computing device for each educator and student on an immersed campus to 
ensure on-demand access to technology; 

• Productivity, communication, and presentation software for use as learning tools; 

• Online instructional resources that support the state curriculum in English language aits, 
mathematics, science, and social studies; 

• Online assessment tools to diagnose students’ strengths and weaknesses or to assess their 
progress in mastery of the core curriculum; 

• Professional development for teachers to help them integrate technology into teaching, learning, 
and the curriculum; and 

• Initial and ongoing technical support for all parts of the package. 

Through an expert review process, the TEA selected three lead vendors as providers of technology 
immersion packages (Dell Computer Inc., Apple Computer Inc., and Region 1 Education Service Center 
[ESC]). Package costs, which ranged from about $1,100 to $1,600 per student, varied according to the 
numbers of students and teachers, the type of laptop computer, and the vendor provider. Of the 21 
immersion sites studied in the second and third years, 5 middle schools selected the Apple package, 15 
selected the Dell package, and 1 school selected the Region 1 ESC package (Dell computer). 

Table 3.1 provides an overview of the basic components within each package and the individual vendors 
that provided various products. All vendors offered a wireless laptop as the mobile computing device 
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(Apple or Dell), and all laptops had a suite of productivity tools (either AppleWorks or Microsoft Office). 
Dell computers also had a web-based portal ( eChalk ) to applications and resources. 


Table 3.1. Technology Immersion Packages 


Component 

Apple 

N = 5 Schools 

Dell 

N= 15 Schools 

Region 1 ESC 
N= 1 School 

Wireless laptop computer 

Apple 
iBook G4 

Dell Inspiron 
or Latitude 

Dell 

Inspiron 

Productivity software 

AppleWorks 

MS Office 
eChalk 

MS Office 
eChalk 

Online resources 

Various 

Various 

Various 

Online assessment 

AssessmentMaster 

i-Know 

i-Know 

Professional development 

Apple Model 

Pearson Achievement, 
Dell Exchange 

ESC 1 — 

Classroom Connect 

Technical and pedagogical 
support 

Apple, 

Campus/District 

Dell, - 

Campus/District 

ESC 1 — 
Campus/District 


Immersion packages also included a variety of digital resources. Apple provided netTrekker, ClassTooIs 
Math , ExploreLearning Math and Science, TeenBizSOOO, and My Access Writing. Dell provided 
netTrekker and Connected Tech, and Region 1 ESC provided Connected Tech, Unitedstreaming, 
Encyclopedia Britannica, EBSCO, NewsBank, and KI2 Teaching and Learning Center. Packages also 
included formative assessments ( AssessmentMaster or i-Know). Additionally, each vendor provided 
professional development as well as ongoing technical support. Apple had its own professional 
development model. Dell relied on a commercial provider ( Pearson Achievement Solutions) and the Dell 
Exchange (an online resource). Region 1 ESC used a combination of service center support plus services 
offered through Connected Coaching and Connected University. (See Appendix D for a more 
comprehensive description of the package components.) 

During the third implementation year, schools began to selectively purchase online resources and 
assessments according to their perceived needs. For example, some schools dropped the online 
assessments because they had state-provided or local assessments that filled their testing needs. Two 
schools (with Dell and ESC lpackages) purchased the My Access Writing program included in the Apple 
package. Schools and teachers also continued in the third year to supplement package resources with 
products purchased locally, provided through state textbook adoptions, or obtained from the Internet free 
of charge. 


Measuring Implementation Fidelity 

Implementation is measured as the fidelity with which technology immersion components and related 
elements attain an envisioned “ideal.” This approach involves gathering extensive data on immersion 
components at each of the treatment campuses and comparing campus-to-campus variations with the 
vision for “full” implementation. The seven immersion components include five supports for 
implementation (Leadership, Teacher Support, Parent and Community Support, Technical Support, and 
Professional Development) and two components related to teacher and student implementation outcomes 
(Classroom Immersion and Student Access and Use). Consistent with second-year procedures, we used a 
two-part measurement approach in the third year. First, we used indicators to describe each campus’ 
progress on a 4-stage scale toward immersion standards. Rating scales for components and related 
elements identify four levels of immersion: minimal (0 to 1.99), partial (2.00 to 2.99), substantial (3.00 to 
3.49), and full (3.50 to 4.00). Second, we used quantitative implementation indices to gauge the level of 
technology immersion using standardized scores (z scores). Z scores allowed the calculation of composite 
scores across indicators with varying scales and standard deviations. 
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Implementation Indicators 


Both the immersion standard scores and implementation indices are derived from values for the seven 
components and their related elements. Scores come from spring 2007 surveys of teachers (N — 619, 
including 371 core-subject teachers) and students (77=6,634) at treatment schools. Table 3.2 provides 
descriptions of the technology immersion indicators. Appendix D provides additional technical detail on 
the measurement of implementation fidelity and the scoring rubrics that describe the four levels of 
immersion. 

Table 3.2. Description of Implementation Indicators for Technology Immersion 

Support for Technology Immersion 
Leadership 

To what extent do teachers indicate that administrators establish a clear vision and expectations, encourage integration, 
provide supports, and involve staff in making decisions about instructional technology. 

Teacher Support 

To what extent do teachers share an understanding about technology use, do teachers continually learn and seek new 
ideas, are teachers unafraid to learn about and use technologies, and are teachers supportive of integration efforts. 

Parent and Community Support 

To what extent do teachers believe that parents and the surrounding community support the school’s efforts with 
technology. 

Technical Support 

To what extent do teachers indicate that technical problems with computers, Internet access, repairs, and material 
availability pose barriers to technology immersion. 

Professional Development 

Contact Hours: To what extent does the duration (hours) of technology-related professional development (PD) support 
the integration of technology into teaching, lear ning, and the curriculum. 

Classroom Support: To what extent do core-subject teachers receive coaching or mentoring from an internal source, 
such as another teacher or technology coordinator, or an external (non-school) source. 

Content Focus: To what extent do core-subject teachers indicate that PD emphasizes curriculum, instructional 
methods, and lesson development in core subjects. 

Coherence: To what extent do core-subject teachers indicate that PD is consistent with personal and school goals, 
builds on prior learning, and supports state standards and assessments. 

Classroom Immersion 

Technology Integration: To what extent do core teachers alter instructional practices, allocate time, integrate research 
on teaching and learning, improve basic skills, and support higher order thinking through technology. 

Learner- Centered Instruction: To what extent do teachers have students establish learning goals, use information and 
inquiry skills, complete alternative assessments, and have active and relevant learning experiences. 

Student Classroom Activities: To what extent do teachers have students use particular technology resources for 
learning in core-subject classes, such as a word processor for writing, a spreadsheet for calculation or graphing, or the 
Internet for research. 

Communication: To what extent do teachers use technology to communicate with students, parents, and colleagues or 
to post information on a class website. 

Professional Productivity: To what extent do teachers use technology to enhance their professional productivity (e.g., 
keep records, analyze data, develop lessons, deliver information). 

Student Access and Use 

Laptop Access: To what extent do students have access to wireless Japtops throughout the school year. 

Core-Subject Learning: How frequently do students use technology resources for learning in core-subject classes. 
Home Learning: To what extent do students have access to and use laptops outside of the school for homework and 

learning. 

Note. See Appendix D for a technical description of the measurement of implementation indicators. 
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Computing Implementation Scores 
Scores for Immersion Standards 

We used teacher and student survey data to compute implementation scores for indicators that measured 
progress toward immersion standards (i.e., minimal to full implementation). Adapting a process 
developed by the RAND Corporation, 1 the value for each indicator was computed relative to the 
maximum value (4.00 — the value assigned to full implementation). Standardization based on the 
maximum value allowed comparisons across different types of indicators. For each component and 
element of technology immersion, standardization involved the following computations: 

• Agreement scales (i.e., strongly agree or strongly disagree with a prescribed practice or 
behavior): 4 = strongly agree, 3 = agree, 2 = neither agree nor disagree, 1 = disagree, and 
0 = strongly disagree. 

• Frequency scales (i.e., four- or five-level frequencies of doing a prescribed practice): 

4 = highest frequency met, 3 or 2.67 = second highest frequency, 2 or 1.33 = third-highest 
frequency, 1 = fourth-highest frequency, and 0 = never or do not do. 

• Continuous variables (i.e., how much time or how often a prescribed practice is done): 

4 = meet or exceed requirements, and 0-3.99 = proportional fraction of requirement. 

Scores for Implementation Indices 

In addition to the standards-based scoring system described above, we used teacher and student survey 
data to compute standardized implementation indicators (z scores with a mean of 0 and standard deviation 
of 1.0) that could then be aggregated to generate: 

• A single implementation score for each technology immersion component for each school 
(e.g.. Leadership Index), 

• A mean implementation support score for the five support components (Support Index), 
including Leadership, Teacher Support, Parent and Community Support, Technical Support, 
and Professional Development, and 

• an overall mean implementation score for each school (Implementation Index), which is an 
average of the Support Index, Classroom Immersion Index, and Student Access and Use 
Index. 2 


Implementation of Technology Immersion 

The sections to follow present findings on (a) the extent to which schools provided the implementation 
supports considered essential to advance technology immersion, and (b) the degree to which schools 
implemented components relevant to teachers’ classroom immersion practices and students’ technology 
access and use. We first present results for implementation standards (measured at four levels) that 
describe the extent to which the model’s support components and instructional and learning components 
are implemented as designed. These scores show whether middle schools are attaining the standards that 
represent what a substantially or fully immersed campus should achieve. Next, we use implementation 


1 Vernez, G., Karam, R., Mariano, L.T., & DeMartini, C. (2006). Evaluating Comprehensive School Reform Models at Scale: 
Focus on Implementation. Santa Monica, CA: RAND. 

2 

“ Variables were standardized as z scores from their original scale or continuous variable values. The use of z scores rather than 
the immersion standard scores was necessary in order to aggregate data across variables that had widely varying standard 
deviations. 
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indices (z scores) to provide an overall measure of technology immersion (Implementation Index) and to 
compare the relative level of implementation for components across campuses. 

Implementation Standards 

As explained previously, progress toward technology immersion standards is measured at four levels 
0 minimal , 0-1.99; partial, 2.00-2.99; substantial, 3.00-3.49; and full immersion, 3.50-4.00) across seven 
components. Five components assess the strength of supports for technology immersion (Leadership, 
Teacher Support, Parent/Community Support, Technical Support, Professional Development), whereas 
one component gauges the extent of teachers’ Classroom Immersion and another component measures 
Student Access and Use (of technology). Figure 3.1 displays the mean implementation scores by 
component and project year. 


4 i 



□ Year 2 BYear3 


Figure 3.1. Mean level of implementation (measured on a 0 to 4 scale) for seven Technology 
Immersion components (N=21 middle schools) by year. 3 

Mean immersion standard scores showed small increases between the second implementation year (2005- 
06) and the third year (2006-07) for all components except Student Access and Use. Despite progress, 
however, third-year mean scores ranging from 2.51 (Professional Development) to 3.14 (Teacher 
Support) showed that supports for technology immersion from school administrators, teachers, the 
community, technical staff, and professional development providers did not meet full implementation 
standards (mean score of 3.50 to 4.00). And, consistent with the second year, teachers, on average, 
reported only partial levels of Classroom Immersion (M = 2.60) in the third year, and students, as a 
whole, reported partial levels of technology access and use (. M - 2.15). Results for individual 
components are discussed in detail below. 


3 Standards-based scores for Professional Development, Classroom Immersion, and Student Access and Use are averages across 
elements of these components. These scores serve descriptive purposes. Composite z scores are used in statistical analyses. 
Student Access and Use data are for 19 middle schools. 
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Level of Principal, Teacher, and Parent/Community Support 

The technology immersion model calls for the systemic integration of technology into all aspects of the 
school. Momentum for implementation, thus, depends upon the backing and support of individuals, 
establishment of institutional norms, and assistance from the surrounding community. Sections to follow 
describe teachers’ reported support from key constituents. 

Leadership. Administrators play key roles in setting the direction for technology immersion, providing 
resources, and building the capacity of staff. Thus, teachers at each school have been asked every year to 
rate the quality of administrative leadership. Administrators demonstrated leadership through behaviors 
such as involving staff in decisions, setting clear expectations for technology use, encouraging and 
participating in professional development events, and providing resources and support. Results in Figure 
3.2 show that administrative leadership was stable across the second and third implementation years. 
Teachers at nearly half of campuses reported substantial levels of leadership, with mean scores across 
years (3.19 and 3.25, respectively) indicating that they either agreed or strongly agreed that 
administrators provided technology-related leadership. Teachers in an additional half of schools reported 
partial levels of administrative support (M = 2.64 and 2.69, respectively). No schools had the level of 
leadership required for full immersion. 



□ Minimal □ Partial ■ Substantial □Full 


Figure 3.2. Level of implementation (measured on a 0 to 4 scale) for Leadership, by the mean 
implementation score and percentage of schools at each implementation level for years 2 and 3. 

Teacher Support. Teacher “buy-in” for technology immersion is critically important because students’ 
school experiences with technology are largely dictated by their teachers. Thus, it is noteworthy that 
teachers reported increased levels of support for technology immersion across years (Figure 3.3). In the 
third year, teachers at about a tenth of campuses reported a full level of support (M = 3.66). That is, 
teachers at these schools strongly agreed that they shared an understanding about technology use for 
student learning, were continually learning and seeking new ideas, were not afraid to learn about and use 
new technologies, and were supportive of integration efforts. Teachers at nearly two-thirds of schools 
reported a substantial level of support for technology innovation ( M = 3.20). In contrast, teachers at about 
a quarter of campuses reported just partial levels of support (M= 2.83). 
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Figure 3.3. Level of implementation (measured on a 0 to 4 scale) for Teacher Support, by the mean 
implementation score and percentage of schools at each implementation level for years 2 and 3. 

Parent and Community Support. Since parents must share responsibility for an expensive laptop 
computer with their child or children, their understanding of and support for technology immersion is 
imperative. Additionally, the enthusiastic support of community members, including elected members of 
the local school board and business people, may influence implementation through mechanisms such as 
the adoption of supportive policies, provision of resources, or promotion of positive public relations. 
Given the importance of parent and community support, teachers’ increased perceptions of such support 
across years were important (Figure 3.4). In the third year, teachers at more than one-third of schools 
reported substantial levels of parent and community support (M = 3.24), with teachers generally agreeing 
that parents and the surrounding community supported their efforts with technology. Although teachers at 
almost two-thirds of schools reported just partial levels of parent and community support ( M = 2.55), this 
represented an improvement over the second year when teachers at three-quarters of schools reported only 
minimal to partial levels of support. 



Figure 3.4. Level of implementation (measured on a 0 to 4 scale) for Parent and Community Support, 
by the mean implementation score and percentage of schools at each implementation level for years 
2 and 3. 
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Level of Technical and Pedagogical Support 

Technical and pedagogical supports are critical aspects of the technology immersion model. As schools 
build their network infrastructure and acquire computer hardware and technology resources, ongoing 
technical support for all components of immersion and ongoing professional development in integrating 
technology into teaching and learning are essential for successful implementation. 

Technical Support. Technical support for immersion should be provided by vendor technicians as well 
as district and campus staff who assist with implementation and offer timely support when technical 
problems arise. Similar to other support mechanisms described above, the level of technical support 
improved between the second and third project years (see Figure 3.5). Although teachers at more than a 
fourth of schools reported substantial levels of technical support in the third year (M = 3.17), teachers at 
nearly three-fourths of schools reported just a partial level of technical support (M — 2.68). Teachers at 
schools with partial implementation were generally unsure that school computers are kept in working 
order, requests for assistance are addressed in a timely way, Internet connections work adequately, and 
classroom materials are readily available. Findings as a whole suggested that technical problems 
continued to challenge many teachers in the third year. 



□ Minimal □ Partial □ Substantial ■ Full 


Figure 3.5. Level of implementation (measured on a 0 to 4 scale) for Technical Support, by the mean 
implementation score and percentage of schools at each implementation level for years 2 and 3. 

Professional Development. Each of the technology immersion packages included a professional 
development component designed to support all educators on an implementing campus. The immersion 
model requires professional development that instructs teachers in effective classroom integration and is 
delivered through proven methods (i.e., learning through a variety of delivery systems, collaboration, 
sustained learning opportunities, and ongoing coaching and support). Although professional development 
providers arc obligated to support all teachers, our implementation measure concentrates on core-subject 
teachers because of their close association with measured student academic outcomes. Year-to-year 
comparisons displayed in Figure 3.6 for the composite Professional Development indicator (mean score 
for the standards-based elements) show there was little difference in the levels of implementation between 
the second and third years. The majority of campuses had minimal to partial levels of implementation for 
professional development across years 2 and 3. About a fifth of campuses achieved a substantial level of 
professional development in the third year (M = 3.09). 
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□ Minimal □ Partial □ Substantial ■ Full 


Figure 3.6. Level of implementation (measured on a 0 to 4 scale) for Professional Development, by 
the mean implementation score and percentage of schools at each implementation level for years 
2 and 3. 

Figure 3.7 compares the implementation levels for each of the elements that contributed to the composite 
Professional Development measures. Mean immersion standard scores at the partial implementation level 
across years indicate that core teachers did not receive either the targeted amount or type of professional 
development. 
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Contact Hours Classroom Support Content Focus Coherence 


□ Year 2 ■ Year 3 


Figure 3.7. Level of Implementation (measured on a 0 to 4 scale) for elements of the Professional 
Development component (Contact Hours, Classroom Support, Content Focus, and Coherence) by 
mean implementation score and year. 

First, teachers reported receiving less than the prescribed number of hours of technology-related 
professional development for each implementation year (estimated to be about 40 to 56 hours per year). 
Mean implementation scores (2.30 and 2.53) indicated that teachers, on average, participated in 30 hours 
or less of technology-related professional development each year. Additionally, teachers reported that 
they received just partial levels of classroom support for technology immersion. Mean scores (2.13 and 
2. 14 across years) meant that teachers as a whole rarely (a few times a year) or never received classroom 
coaching or mentoring from an internal source (such as another teacher or technology coordinator) or 
external source (such as a vendor-provided professional trainer). 

Core-subject teachers who participated in technology-related professional development also expressed 
uncertainty about the extent to which activities supported their curricular and instructional goals. 
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Teachers, on average, reported that the content of professional development placed a minor emphasis on 
curriculum, instructional methods, and lesson development in core areas (mean scores of 2.97 and 2.89 
reflect partial implementation). Additionally, teachers as a whole failed to see the coherence of 
technology-related professional development with their personal goals, earlier learning experiences, and 
state/district curriculum standards and assessments. Teachers’ mean ratings across years (2.54 and 2.49) 
indicated that professional development was coherent to a minimal extent (partial implementation). 

Level of Classroom Immersion 

Given the needed equipment, digital resources, and support for technology immersion, teachers are 
expected to design technology-enhanced learning environments and integrate technology into teaching, 
learning, and the curriculum. Teachers’ composite level of Classroom Immersion across project years 
indicates that teachers at a few schools made progress in creating technology-immersed classrooms 
(Figure 3.8). Teachers at about a tenth of schools had a substantial level of classroom immersion in the 
third year ( M =3.09). In contrast, teachers at the majority of schools reported only partial levels of 
Classroom Immersion in both the second (M = 2.45) and third project years (M =2.47). 



□ Minimal □ Partial □ Substantial ■ Full 


Figure 3.8. Level of implementation (measured on a 0 to 4 scale) for the Classroom Immersion, by 
the mean implementation score, percentage of schools at each implementation level, and year. 

Figure 3.9 illustrates teachers’ level of implementation relative to five elements of Classroom Immersion: 
Technology Integration, Learner-Centered Instruction, Student Classroom Activities (with technology), 
Communication, and Professional Productivity. On average, teachers reported partial levels of 
implementation for the elements of Classroom Immersion. However, for each element, teachers reported 
slightly stronger implementation in the third year. The largest increases were for teachers’ ideological 
affiliations with Technology Integration and Learner-Centered Instruction, whereas the smallest change 
involved Student Activities with technology in classrooms. Thus, teachers became somewhat more 
positive about technology integration and constructivist methods but changed beliefs did not necessarily 
increase the frequency of classroom activities. On the other hand, teachers reported notable increases in 
the third year relative to their use of technology as a communication tool and for the enhancement of their 
professional productivity. 
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Figure 3.9. Level of Implementation (measured on a 0 to 4 scale) for elements of Classroom 
Immersion (Technology Integration, Learner-Centered Instruction, Student Activities, 
Communication, and Professional Productivity) by mean implementation score and year. 

Level of Student Technology Access and Use 

The transformation of classroom experiences is a vital part of technology immersion, but the model also 
aims for students to have on-demand technology access both within and outside of school that allows 
them to become more independent and self-determined learners. Overall, data reported by students 
indicated that Student Access and Use remained relatively stable across the second and third years (Figure 
3.10). Students at more than two-thirds of schools had partial access and use (mean implementation levels 
of 2.38 and 2.35 across years), whereas students at about a third of schools had minimal access and use 
(mean implementation levels of 1.60 and 1.74, respectively). 



□ Minimal □Partial □ Substantial ■Full 


Figure 3.10. Level of implementation (measured on a 0 to 4 scale) for the Student Access and Use, 
by the mean implementation score and percentage of schools at each implementation level for years 
2 and 3. 

Figure 3.1 1 shows the average level of implementation for three elements of Student Access and Use: 
Laptop Access Days, Core-Content Learning, and Home Learning. First, in a fully immersed school, all 
students should have access to their wireless laptops and resources nearly the entire school year (about 
170 to 180 days). Schools as a whole, however, had difficulty keeping laptops in the hands of students. 
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Year-to-year comparisons indicated that the mean implementation level for Laptop Access Days declined 
between the second and third years (from 2.69 to 2.50). Thus, students, on average, had laptops for a 
smaller number of days in the third year. Partial levels of implementation indicate that student access days 
typically varied at schools to a large extent (from 100 to 176 days per student). However, Laptop Access 
Days also varied widely across schools. In the third year, students at 16% of schools reported either 
substantial or full laptop access. In contrast, students at 63% of schools reported partial access, and 
students at 21% of schools reported minimal laptop access. 
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□ Year 2 BYear3 

Figure 3.11. Level of Implementation (measured on a 0 to 4 scale) for elements of Student Access 
and Use (Laptop Access Days, Core-Content Learning, and Home Learning) by mean 
implementation score and year. 

Students also estimated how often they used their laptops in their English/language arts, mathematics, 
science, and social studies classes and for learning at home. In contrast to Laptop Access Days, there were 
small increases in the third year for both Core-Content Learning and Home Learning. Still, students as a 
whole reported just a partial level of implementation across years for Core-Content Learning (M= 2.07 
and 2.12), with laptops typically used sometimes (once or twice a month) to often (once or twice a week) 
in classrooms. Students, on average, used their laptop even less frequently for learning outside of school. 
Students reported a minimal level of laptop use for home learning in both the second and third years (M = 
1.75 and 1.84). Thus, students as a whole used their laptops outside of school for homework and learning 
either not at all or to a trivial extent. 

Overall, students’ opportunities to use their laptops both within classrooms and outside of school were 
affected by the number of days that students actually had their laptops. In some schools, students’ laptop 
access days were drastically reduced by factors such as time for repairs, technical issues, disciplinary 
infractions, and parent resistance. Students in other schools, contrary to the tenets of technology 
immersion, were not allowed to take their laptops home, or their home use was restricted in some way 
(e.g., laptops could only be used for special assignments). 

In sum, overall results for the implementation of technology immersion as measured by standards-based 
scores suggest that the levels of support for implementation increased to some extent between the second 
and third years. Similarly, teachers’ reported a slightly increased level of Classroom Immersion. In 
contrast, the level of Student Access and Use was relatively stable across years. Findings for standards- 
based scores also show that the level of implementation varied by campus. By the end of the third year, 
none of the middle schools achieved ///// immersion. Evidence suggests that a few campuses reached 
substantial immersion, whereas the majority of schools achieved only minimal to partial immersion. 
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Implementation Indices 


To further illustrate each school’s level of immersion in the third year, Table 3.3 presents the composite 
campus Implementation Index (z score) alongside implementation indices (z scores) for each of the seven 
components. Z scores have a mean of 0 and a standard deviation of 1.0. Thus, the campus score indicates 
how many standard deviations from the mean a score lies. Schools with scores above 0 have higher values 
on the components of technology immersion, whereas schools with index values below 0 show less 
evidence of the immersion. The Implementation Index is an average score for the Support Index, 
Classroom Immersion Index, and Student Access and Use Index. 4 

Table 3.3. Third-Year Implementation of Technology Immersion 

j Support Index j 


Middle 

School 

(MS) 

Leader- 

ship 

Index 

Teacher 

Support 

Index 

Parent/ 

Comm. 

Index 

Technical 

Support 

Index 

PD 

Index 

Classroom 

Immersion 

Index 

Student 

Access/Use 

Index 

Implemeii' 

tation 

Index 

MS 1 

1.29 

1.82 

1.17 

0.94 

1.79 

1.92 

0.83 

1.89 

MS 2 

1.59 

2.09 

1.40 

1.78 

1.72 

0.24 

1.38 

1.58 

MS 3 

0.22 

0.12 

-0.75 

0.39 

0.85 

1.66 

1.39 

1.35 

MS 4 

0.97 

0.75 

0.90 

0.47 

0.27 

0.78 

0.43 

0.87 

MS 5 

0.92 

0.62 

0.97 

-0.03 

0.93 

0.63 

0.22 

0.72 

MS 6 

1.02 

0.98 

0.46 

0.96 

-0.62 

-0.42 

0.90 

0.50 

MS 7 

0.88 

0.41 

0.34 

0.42 

1.03 

1.13 

-0.77 

0.49 

MS 8 

1.05 

0.71 

-0.13 

-0.29 

-0.37 

-0.26 

-0.04 

0.20 

MS 9 

0.60 

0.21 

0.23 

-1.56 

0.63 

1.03 

-0.90 

0.08 

MS 10 

-1.64 

-0.99 

-0.46 

0.17 

-1.01 

-0.42 

1.19 

-0.07 

MS 11 

0.00 

0.13 

0.58 

-0.26 

0.75 

-0.46 

-0.46 

-0.25 

MS 12 

-0.17 

-0.28 

-0.90 

0.71 

-0.25 

-0.14 

-0.36 

-0.29 

MS 13 

-0.61 

-1.41 

-0.51 

-1.61 

-0.78 

-1.09 

0.91 

-0.58 

MS 14 

-1.29 

-0.41 

-1.55 

-0.86 

-1.25 

-0.45 

0.38 

-0.58 

MS 15 

-0.69 

0.82 

1.38 

0.94 

-1.33 

-2.40 

0.53 

-0.66 

MS 16 

-0.37 

-0.96 

-1.11 

-1.66 

0.39 

-0.31 

-1.65 

-1.20 

MS 17 

-1.93 

-1.41 

-0.99 

-1.45 

-0.16 

-0.50 

-1.03 

-1.25 

MS 18 

-0.59 

-0.52 

-1.68 

-0.49 

-0.63 

-0.90 

-1.19 

-1.27 

MS 19 

-0.85 

-1.08 

-0.89 

-0.13 

-1.80 

-0.84 

-1.71 

-1.55 

MS 20 

-0.66 

-0.52 

1.48 

0.15 

0.52 

0.61 

— 

a 

MS 21 

0.25 

-1.08 

0.09 

1.41 

-0.66 

-0.33 

— 

a 


Note. Implementation indices are z scores with a mean of 0 and a standard deviation of 1 .0. Scores above zero indicate a 
greater presence of technology immersion components and higher levels of implementation. 
a School did not submit spring 2007 student surveys. 


Despite some variations in component scores, middle schools with higher values on the Implementation 
Index tended to have component scores that indicated a stronger presence of the immersion attributes 
such as administrative leadership and teacher support for immersion. In contrast, middle schools that had 
more negative values on the Implementation Index generally had negative values for nearly all of the 
immersion components. These findings suggest that the implementation indices are relatively effective in 
discriminating higher and lower implementing schools. Still, there are exceptions to the prevailing trends. 
Some schools, such as MS 3, had generally higher implementation values for most of the indicators 
except Parent/Community Support (-0.75). This suggests that gaining full parent support was a problem 
for that school in the third year. In other schools, such as MS 13, students reported higher levels of 


4 Two schools did not submit spring 2007 student surveys, so those schools do not have scores for Student Access and Use or the 
Implementation Index. 
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technology access and use even though strong implementation supports were not in place, and their 
teachers’ level of Classroom Immersion was low. 

Campus-level results for the Implementation Index displayed in Figure 3.12 illustrate the variation in the 
levels of technology immersion for the 19 middle schools with scores for the third project year. Results 
for the Implementation Index combined with evidence from standards-based scores suggest that about a 
quarter of middle schools (5), with Implementation Index scores ranging from 0.72 to 1.89 standard 
deviations above the mean, have a stronger presence of the components of technology immersion 
compared to other schools, thus a higher level of implementation that more nearly approximates expected 
standards. 



□ Implementation Index, Year 3 


Figure 3.12. Campus means for 19 immersion middle schools (MS) on the Technology Immersion 
Implementation Index (standardized scores [z scores] with a mean of 0 and a standard deviation of 1.0). 

Conclusions 

This chapter described the components of technology immersion, as defined by the TEA and 
operationalized through technology immersion packages. In the second and third project years, we 
measured implementation using a two-part approach: (a) designation of standards defining four levels of 
immersion ( minimal , partial, substantial, and full), and (b) calculation of standardized implementation 
indices (z scores). Both types of scores are derived from values for components relative to supports for 
immersion, and the extent of classroom immersion and students’ technology access and use. Major 
findings are the following. 

• Mean immersion standard scores revealed small increases between the second implementation 
year (2005-06) and the third year (2006-07) for each of the support components (Leadership, 
Teacher Support, Parent/Community Support, Technical Support, and Professional Development) 
as well as teachers’ overall level of Classroom Immersion. In contrast, the level of Student 
Access and Use was stable across years. 

• Despite improvements, mean third-year immersion standard scores (ranging from 2.51 to 3.14) 
showed that many schools needed stronger supports, especially in the areas of school leadership, 
parent and community support for technology use, technical supports that addressed obstacles to 
technology use, and professional development. 
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• Consistent with the second project year, core-subject teachers at the majority of schools reported 
only partial levels of Classroom Immersion in the third year. Teachers’ mean scores at a few 
schools, however, revealed progress in creating technology-immersed classrooms. Accordingly, 
the standards-based implementation score for Classroom Immersion increased slightly across 
years (from 2.48 to 2.60). 

• Immersion standard scores for each of five elements of Classroom Immersion showed slightly 
stronger implementation in the third year, with the largest increases for teachers’ ideological 
affiliations with Technology Integration and Learner-Centered Instruction, and the smallest 
change for Student Activities with technology in the classroom. There were notable increases in 
teachers’ use of technology as a communication tool and for the enhancement of their own 
professional productivity in the third year. 

• Students’ access to and use of laptops for learning within and outside of school generally fell 
short of expectations in the third year. Students at more than two-thirds of schools had partial 
levels of access and use (mean immersion standard scores of 2.38 and 2.35 across years), whereas 
students at about a third of schools had only minimal access and use (mean immersion scores of 
1.60 and 1.74 across years). 

• Students’ opportunities to use their laptops both within classrooms and outside of school were 
affected by the number of days that students actually had their laptops. Year-to-year comparisons 
indicated that the mean implementation level for Laptop Access Days declined between the 
second and third project years (from 2.69 to 2.50). Partial levels of implementation indicated that 
student access days generally varied at schools to a large extent (from 100 to 176 days per 
student). In contrast to laptop days, students reported small increases in the third year in their use 
of laptops for Core-Content Learning and Home Learning. 

• Implementation indices (z scores) described each school’s level of implementation for the 
components of technology immersion. Third-year evidence from immersion standard scores and 
the Implementation Index, a composite score measuring the overall presence of immersion 
components, indicated that about a quarter of middle schools (5) had a much stronger presence of 
the immersion components compared to other schools. Thus, these schools had a higher level of 
immersion that more nearly approximated expected implementation standards. 

Despite low levels of implementation at many campuses, report chapters to follow demonstrate that 
technology immersion can positively affect teachers and students in many ways even at lower 
implementation stages. 
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4. Effects of Technology Immersion on Teachers and Teaching 


In the theoretical model, researchers hypothesize that quality implementation of technology immersion 
(i.e., supportive school leaders, teacher commitment to technology innovation, professional 
development supporting curricular integration, adequate technical and pedagogical support to maintain 
an immersed campus, and robust student technology access) should lead to teachers who have greater 
technology proficiency, use technology more for their own professional productivity, hold a more 
favorable pedagogical orientation toward technology, and collaborate more often with their peers to 
advance teaching and learning through technology. Moreover, teachers in schools that achieve higher 
levels of school and classroom immersion will use laptops as a tool to increase the intellectual 
challenge of lessons and will have students who use technology more often in their classrooms. 

Contrary to expectations, results reported in Chapter 3 revealed that school-level supports for 
technology immersion generally did not meet full immersion standards, and accordingly, teachers at 
the majority of schools reported just partial levels of Classroom Immersion. Additionally, as noted in 
the methodology, control schools began to plan for technology immersion in the third year. Thus, most 
of the control teachers received new laptops and instructional resources along with increased 
opportunities for technology-related professional development. Recognizing that these factors may 
influence outcomes, we have investigated the effect of technology immersion on teachers, given that 
the level of implementation at treatment schools was generally low and control teachers began to 
benefit from elements of the treatment. 

Findings on the effects of technology immersion on teacher-mediating variables come from online 
surveys of teachers completed in fall 2004 ( N - 1,271) and again in spring of 2005 ( N = 1,144), 2006 
N = 1,175), and 2007 ( N = 1,208). Teachers responded to items pertaining to their personal and 
classroom technology experiences. Response rates ranged from 87% to 98% across survey 
administrations, with only small differences between comparison groups. Surveys included measures 
of seven teacher-level variables. Teachers responded to items gauging their technology knowledge and 
skills (Technology Proficiency and Professional Productivity); the strength of their ideological views 
relative to Technology Integration, Learner-Centered Instruction, and Resistance to Integration; the 
frequency of Student Classroom Activities with technology; and their Collaboration with peers on 
technology. Cronbach’s alpha reliability coefficients for the teacher-level scale scores ranged from 
0.66 to 0.98. (See Appendix C for technical details.) 

Researchers also conducted classroom observations during site visits at each of the treatment and 
control schools to gather information on instructional practices and changes across time. Classroom 
observations focused across incrementally on the teachers of Cohorts 1, 2, and 3 students. We 
conducted observations in a sample of sixth-grade, core-subject classrooms in fall 2004 and again in 
spring 2005. In spring 2006, researchers observed in sixth- and seventh-grade classrooms. In spring 
2007, researchers observed in sixth-, seventh-, and eighth-grade classrooms. 

Teacher Mediating Variables — HLM Analysis 

One advantage of a longitudinal study is the potential to study the nature of teacher change. The 
development of hierarchical linear models (HLM) has provided statistical tools for studying rates of 
change using measurements from multiple time points (Raudenbush & Bryk, 2002). For this study, we 
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measured teacher variables on four occasions: fall 2004 (baseline), spring 2005 (after the first 
implementation year), spring 2006 (after the second year), and spring 2007 (after the third year). Our 
analytical sample included 1,684 teachers who taught at schools at some point during three 
implementation years, with 816 in 21 technology immersion schools and 868 in 21 control schools. 
Thus, we included teachers in the analyses even if they were not measured at all four time points. 
Because multilevel regression models do not assume equal numbers of observations (i.e., occasions of 
measurement), respondents with missing data can remain in the analysis (Hedeker, 2004; Hox, 2002). 
HLM, however, requires complete data at the teacher and school levels, so teachers were omitted if, 
for example, they were missing demographic information such as ethnicity. Our analytic approach 
mitigated problems associated with the substantial loss of teachers from analyses due to generally high 
teacher attrition rates each year of the study and varying teacher turnover rates across schools. For 
example, while the annual average teacher turnover rate was about 17%, school annual turnover rates 
varied from about 6% to about 42%. 

The analyses that follow contrast immersion and control teachers’ individual growth trajectories for 
each of the seven scales described above. We analyzed effects using three-level hierarchical growth 
models. FILM growth models produce teacher- and school-specific effects (i.e., the extent which the 
survey scores vary across time, teachers, and schools). In our models, we hypothesize that school 
poverty is related to teachers’ initial status and yearly growth rate. This supposition stems from an 
investigation of the implementation of technology immersion indicating that a higher concentration of 
economically disadvantaged students in a school is negatively associated with stronger levels of 
school and classroom immersion. Similarly, other research reviews confirm the negative effects of 
school poverty on school reform efforts (Desimone, 2002) and student achievement (Sirin, 2005). 

Since Technology Immersion Pilot (TIP) grants targeted high-needs schools, the percentages of 
disadvantaged students were generally high across most of the study’s schools. Even so, school 
poverty concentrations varied substantially (ranging from 31% to 100%). The statistical model is 
described below. 

Level 1 : Repeated-Measures Model 

Level 1 is a repeated-measures model (i.e., survey time within teachers) that enables us to capture key 
features of growth (e.g., initial status, rate of change). In the model, Y nj is the survey scale score at year 
t for teacher / in school j. Survey Time is the point at which teachers completed the online surveys 
(0=fall 2004, I =spring 2005, 2=spring 2006, 3=spring 2007). The key parameters in the model are Koy 
and Kuj. The coefficient noij represents the “initial status” (that is, the initial survey scale score) for 
teacher / in school j in fall 2004, and jr^-is the growth rate (rate of change) for teacher i in school j per 
school year. The e ti j is the error term (within-teacher measurement error) assumed to be normally 
distributed with a mean of 0 and a constant variance. Thus, at level 1 the model is 

Y,ij = Koij + K U j( Survey Time) tij + e tij . 

Level 2: Teacher-Level Model 

The Level 2 model (between-teachers model) allows us to determine differences between teachers in 
features of growth (e.g., initial status, rate of change). In the teacher-level model, jtoy is the teacher’s 
initial survey scale score and nuj is the teacher’s rate of growth per school year. In the model, /3 0 oj 
represents the mean initial status within school j, and p 10j is the mean yearly rate of teacher change 
within school j. The r 0 ij and r Uj arc residuals (i.e., random effects). At level 2, the model is 

KOij = PoOj + r 0ij 

niij=Pioj + r hj. 
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Level 3: School-Level Model 


At the school level (level 3), we examined how teachers’ initial status and growth varied across 
schools as a function of school- level random effects (ju 0 oj and // 10 j) as well as school conditions, 
including immersion status and school poverty. That is, we hypothesized that being in an immersion 
school is positively related to teachers’ growth on technology-related scores, after controlling for the 
poverty level of the school. Thus, we pose the following school-level model: 

Pooj = y 0 oo + Yooi( Immersion Status )j + yoo 2 (School Poverty )j + fiooj 

fiioj — yioo + yioi( Immersion Status)j + y I0 2 (School Poverty)j + /Liioj. 

In the model, fiooj is the mean initial status for teachers in school j and y 0 oo is the overall mean 
initial status (grand mean); f} 10 j is the mean teacher growth rate in school j and y 100 is the overall 
mean teacher growth rate. Immersion status is an indicator variable with a value of 0 for a control 
school and a value of 1 for an immersion school. School poverty is a continuous variable with 
percentages ranging from ranging from 31% to 100%, with a mean of 69.8%. The coefficients 
yooi and y m represent the direction and strength of association of immersion status and school- 
level initial status. 


Effects of Immersion on Teachers 

After adjusting for school poverty, technology immersion had a statistically significant effect on 
teachers’ rates of growth for six technology-related variables (Table 4.1). Teachers at technology 
immersion schools, on average, had significantly steeper growth trends than teachers at control 
schools for Technology Proficiency and Professional Productivity, two measures of teachers’ ideology 
(Technology Integration and Learner-Centered Instruction), the frequency of Student Classroom 
Activities (with technology), and Collaborative interactions with colleagues on technology-related 
issues. In contrast, there was no significant effect of immersion on teachers’ Resistance to Integration. 

Table 4.1. Immersion Effects on Estimated Mean Growth Rates for Teacher Variables 


Statistics for Teachers in Immersion Schools 
with Average School Poverty c 



Immersion 
Effect Net of 
School 
Poverty 

Average 
Estimated 
Initial Status 
Fall 2004 

Yearly 
Growth Rate 

Average 

Estimated 

Score 

Spring 2007 

Yearly 
Growth Rate 
for Control 
Teachers 

Technology Proficiency 11 

Yes 

4.49 

0.31*** 

5.42 

0.13*** 

Professional Productivity 13 

Yes 

2.93 

0.20*** 

3.53 

0.08*** 

Ideology 






Technology Integration 11 

Yes 

3.18 

0.59*** 

4.95 

q 24*** 

Learner-Centered Instruction 11 

Yes 

3.64 

0.38*** 

4.79 

0.20*** 

Resistance to Integration 11 

No 

2.15 

0.02 

2.20 

0.03 

Student Classroom Activities' 3 

Yes 

1.96 

q 23*** 

2.66 

0.03* 

Collaboration 13 

Yes 

2.41 

0.10** 

2.71 

0.04 


Source: Online teacher surveys conducted in fall 2004, spring 2005, spring 2006, and spring 2007. 
*p < .05. **p < .01. ***p < .001. 

‘‘Items measured on a 7-point scale. b Items measured on a 5-point scale. 


Unlike previous years, control teachers also had significantly positive growth trends for technology- 
related variables in the third year. Although control teachers’ yearly growth rates were significantly 
less steep than rates for immersion teachers, the introduction of technology resources in control 
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schools had a positive effect on teachers’ technology proficiencies, ideology, and to a lesser extent, on 
their students’ classroom activities and their collaboration with peers on technology -related issues. 


Figure 4.1 illustrates the estimated growth trajectories for teachers in immersion schools with average 
levels of school poverty. Teachers have positive growth trajectories for all of the technology-related 
indicators, with the exception of Resistance to Integration, which remained stable across years. 



Technology Proficiency (7-pt) 
Productivity (5-pt) 

Technology Integration (7-pt) 
Learner-Centered Instruction (7-pt) 
Resistance to Integration (7-pt) 
Student Classroom Activities (5-pt) 
Collaboration (5-pt) 


Fall 2004 Spring 2005 Spring 2006 Spring 2007 
Survey Time 


Figure 4.1. Estimated mean growth trajectories for Technology Immersion teachers working in 
schools with average levels of school poverty on technology-related indicators (ratings on either 5- 
point or 7-point scales). 


Sections to follow explain changes in teachers’ knowledge and skills, ideology, classroom practices, 
and peer collaboration. Tables 4.2, 4.3, and 4.4 provide school-level statistics for the HLM analyses of 
immersion effects on teacher mediating variables. 


Technology Knowledge and Skills 

Texas Technology Applications Standards require all teachers to master and use technology-related 
terminology, concepts, and strategies, and to use tools to accomplish a range of tasks (e.g., 
communicate with diverse audiences and analyze electronic information). Given the importance of 
teachers’ technology knowledge and skills and the potential impact of immersion, our online surveys 
included measures of teachers’ Technology Proficiency and Professional Productivity. For 
Technology Proficiency, teachers rated their skills in using various technology applications on a 7- 
point scale ranging from 1 (not true of me now ) to 7 (very true of me now). The proficiency scale 
included items measuring technology operations (e.g., send email to coworkers, parents, or peers; 
search for and find a Web site; find primary sources of information on the Internet) and items related 
to classroom instruction (e.g., using the computer for presentations or creating a lesson plan or unit 
incorporating technology). 


HLM statistics in Table 4.2 show that immersion teachers grew in technology proficiency at a 
significantly faster rate (0.31 scale-score point per year) than control teachers (0.13 point per year) 
Immersion teachers began with slightly lower mean proficiency scores than control teachers in fall 
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2004, but they suipassed control teachers in spring 2005 and continued to widen the proficiency gap 
during the next two school years. 


Table 4.2. Immersion (Fixed) Effect Analyses for Teacher Technology 
Knowledge and Skills Variables 


Dependent variable 
and predictor 

Technology Proficiency 
Gamma 

Coefficient r- value 

Professional 

Gamma 

Coefficient 

Productivity 
1- value 

Initial status (fall 2004) 

4.673 

54.61*** 

3.023 

49.94*** 

Immersion 

-0.184 

-1.52 

-0.094 

-1.14 

School Poverty 

-0.001 

-0.40 

0.000 

0.12 

Growth rate 

0.134 

6.35*** 

0.084 

8.16*** 

Immersion 

0.176 

g 72*** 

0.117 

g 17*** 

School Poverty 

-0.002 

-2.44* 

-0.001 

-0.88 


f p < . 10 ; *p < . 05 ; **p < . 01 ; ***p < . 001 . 


Teachers who taught at immersion and control schools with higher levels of school poverty 
(percentages of economically disadvantaged students) had significantly slower rates of growth for 
Technology Proficiency. For each percentage point increase in school poverty, teachers had a 0.002 
scale-score decrease in proficiency. Thus, as Figure 4.2 illustrates, a 20% decrease in school poverty 
predicts a 0.04 point increase in teachers’ yearly growth in proficiency (i.e., 20 x 0.002); a 20% 
increase in school poverty predicts a 0.04 point decrease in teachers’ yearly growth. As the level of 
school poverty increases, the teacher proficiency gap widens. 



• Immersion: Lower School Poverty — ■ — Immersion: Higher School Poverty 

- - A- - Control: Lower School Poverty - - > - Control: Higher School Poverty 


Figure 4.2. Estimated mean growth trajectories for teachers at immersion and control schools for 
Technology Proficiency (ratings on a 7-point scale). Comparisons are for teachers working in 
schools with lower and higher concentrations of school poverty (20% above average and 20% 
below average). 

Teachers also rated the frequency with which they used technology for Professional Productivity on a 
5-point scale ranging from 1 (never) to 5 ( almost daily). Productivity items, for example, measured 
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teachers’ use of technology for administrative, classroom management, communication, and 
instructional puiposes. Similar to findings for Technology Proficiency, teachers at immersion schools 
had significantly steeper rates of growth than control teachers in the use of technology to improve their 
productivity. The estimated yearly mean growth trajectories for immersion and control teachers in 
schools with average poverty were 0.20 and 0.08 scale-score points per year, respectively. Teachers 
working in schools with higher percentages of disadvantaged students grew at slightly slower rates. 

Ideology 

Teachers also responded to items measuring their ideological views relative to technology integration 
and constructivist practices on a 7-point scale, ranging from 1 ( not true of me now) to 7 (very true of 
me now). Items from the Levels of Technology Implementation (LoTi) Questionnaire (Moersch, 2001) 
measured three latent variables (Technology Integration, Learner-Centered Instruction, and Resistance 
to Integration). HLM results detailed in Table 4.3 show that teachers at immersion schools, on 
average, became more positive towards innovative technology practices across time. 


Table 4.3. Immersion (Fixed) Effect Analyses for Teacher Ideology Variables 


Dependent variable 
and predictor 

Technology 

Integration 3 

Learner-Centered 

Instruction 

Resistance 
to Integration 13 

Gamma 

Coefficient 

/-value 

Gamma 

Coefficient 

/-value 

Gamma 

Coefficient 

/-value 

Initial status (fall 2004) 

2.885 

41 52*** 

3.670 

60.79*** 

2.434 

50.38*** 

Immersion 

0.298 

2.89** 

-0.028 

-0.32 

-0.281 

_4 34 *** 

School Poverty 

0.008 

2.72* 

0.006 

2.55* 

-0.272 

-1.72 

Growth rate 

0.237 

7 14*** 

0.198 

7 79 *** 

0.032 

1.85 

Immersion 

0.348 

6.96*** 

0.186 

4.54*** 

-0.017 

-0.58 

School Poverty 

-0.003 

-1.76 

- 0.002 

-1.40 

0.001 

1.12 


*p < .05. **p < .01. ***p < .001. 

d Technology immersion teachers had significantly higher initial Technology Integration scores. A latent variable 
regression, controlling for the effect of this initial difference on the growth rate, showed that original differences were 
statistically insignificant (the difference divided by the standard error of the difference = -0.46). 
b Technology immersion teachers had significantly lower initial Resistance to Integration scores. A latent variable 
regression, controlling for the effect of this initial difference on the growth rate, showed that original differences were 
insignificant (the difference divided by the standard error of the difference = - 0.42). 

The Technology Integration scale included items gauging teachers’ actions supporting curricular and 
instructional infusion of technology. For example, teachers indicated the extent to which computer- 
related activities enabled them to support students’ authentic problem solving or to promote critical 
thinking. Findings show that teachers in immersion schools had a significantly more positive rate of 
change for Technology Integration than control teachers. The mean estimated growth trajectory for 
immersion teachers who worked in schools having average levels of school poverty was 0.59 scale- 
score point per year compared to 0.24 for control teachers. 

Teachers at immersion schools compared to control also changed at a significantly faster rate in their 
affiliations with principles of Learner-Centered Instruction. Across survey administrations, immersion 
teachers reported increasingly higher ratings for items describing pedagogical practices such as having 
students establish individual learning goals, emphasizing experiential learning, and providing real- 
world experiences. The estimated yearly growth in the adoption of learner-centered practices for 
immersion and control teachers in schools with average poverty was 0.38 and 0.20 scale-score points, 
respectively. Teachers in schools with higher concentrations of school poverty had somewhat slower 
rates of growth relative to both technology integration and learner-centered practices. 
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For the Resistance to Integration scale, teachers expressed their strength of association with items 
indicating that classroom computers are not a priority, not a necessary part of instruction, and not 
practical for students. Contrary to the two ideological indicators discussed above, there was little 
change in the growth rate on the Resistance to Integration scale for either immersion or control 
teachers. Scores indicated that teachers, on average, expressed a relatively low level of resistance to 
technology integration, and their level of resistance remained fairly constant across years. 

Student Classroom Activities and Teacher Collaboration 

Table 4.4 provides HLM statistics for measures of teachers’ classroom activities and collegial 
collaboration. The Student Classroom Activities scale provided an estimate of the frequency — on a 5- 
point scale ranging from 1 (never) to 5 ( almost daily ) — with which teachers had students in their 
typical class use technology in various ways. For example, teachers might have students use 
technology for writing, learning and practicing skills, communication, or Internet research. As 
expected, given the availability of laptops at immersion schools, teachers at treatment schools had a 
significantly faster growth rate for Student Classroom Activities (0.23 and 0.03 scale-score points per 
year, respectively, for immersion and control teachers in schools with average poverty). School 
poverty had no discernable effect on teachers’ growth rate for the frequency of students’ classroom 
activities involving technology. Even though immersion teachers had their students use technology in 
classrooms at an increasingly more frequent rate, estimated mean scores indicated that by spring 2007 
teachers, on average, had students use various technology applications in their classes infrequently 
(about once or twice a month, M - 2.65). 


Table 4.4. Immersion (Fixed) Effect Analyses for Student Classroom 
Activities and Teacher Collaboration Variables 


Dependent variable 
and predictor 

Student Classroom 
Activities 

Teacher 

Collaboration 

Gamma 

Coefficient 

f-value 

Gamma 

Coefficient 

1 - value 

Initial status (fall 2004) 

1.888 

40.47*** 

2.300 

44 . 77 *** 

Immersion 

0.075 

1.17 

0.107 

1.49 

School Poverty 

0.004 

2.23* 

0.003 

1.74 

Growth rate 

0.034 

2.36* 

0.035 

1.92 

Immersion 

0.199 

7 43 *** 

0.065 

2 . 20 * 

School Poverty 

- 0.001 

-0.91 

0.000 

0.06 


*p < . 05 . **p < . 01 . ***p < . 001 . 


We also reasoned that a greater abundance of technology resources and opportunities for shared 
professional development would lead to stronger teacher connections. Accordingly, the Collaboration 
scale measured teacher interactions with colleagues that supported improvements in instructional 
practices, such as coaching and mentoring, collectively developing technology lessons, and 
exchanging information about their students. As expected, immersion teachers had a steeper mean 
yearly growth trend for Collaboration (0.10 scale-score point) than control teachers (0.04 point). 
Campus poverty had a negligible effect on teacher collaboration. 

Effects of Immersion on Classroom Practice 

To further understand teachers’ instructional practices, researchers conducted classroom observations 
in samples of core-subject classrooms (reading/English language aits, mathematics, science, and social 
studies) each year. In fall 2004 and spring 2005, we observed sixth-grade teachers of Cohort 1 
students. In spring 2006, the classroom sample included observations of seventh-grade teachers of 
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Cohort 1 students and sixth-grade teachers of Cohort 2 students. In spring 2007, the sample included 
eighth-grade teachers of Cohort 1 students, seventh-grade teachers of Cohort 2 students, and sixth- 
grade teachers of Cohort 3 students. To the extent possible, we conducted follow-up observations for 
teachers who remained at the schools across years. 

Classroom observations involved either single observers (about 75% of classrooms) or pairs of 
observers (about 25% of classrooms). Paired observations permitted the calculation of inter-observer 
agreement. In fall 2004, researchers observed 125 classrooms (60 treatment and 65 control) in half of 
the schools. Subsequently, we conducted observations in all schools. We observed 206 classrooms in 
2005 (105 treatment and 101 control), 217 classrooms in 2006 (114 treatment and 103 control), and 
194 classrooms in 2007 (95 treatment and 99 control). At small campuses, researchers observed nearly 
all core-content teachers; at larger campuses, we observed a representative sample of classrooms. 

Across data-collection periods, observations at treatment and control schools included nearly equal 
proportions of teachers by subject-area taught, gender, highest degree earned, and years teaching 
experience. Observations included somewhat more English language aits and reading teachers (27% to 
30% of observed teachers), and somewhat less mathematics teachers (19% to 28%), social studies 
teachers (17% to 28%), and science teachers (13% to 23%). Variations reflected our interest in 
documenting the instructional practices of teachers whose students were included in cohorts being 
tracked across years and their TAKS-tested subject areas. 

During observations, data collectors used the Observation of Teaching and Learning (OTL) instrument 
to record descriptive information about the classroom environment, and to make time-interval ratings 
for classroom organization, teacher activities and technology use, student activities and technology 
use, student engagement, and student collaboration. Observers also recorded notes during the 
observations to capture the lesson’s content focus and objectives, teachers’ questioning strategies 
(lower and higher order), and students’ learning experiences. Following classroom observations, 
observers used time-interval ratings and descriptive notes to rate the Intellectual Challenge of 
classroom work (rating scales developed by Newmann, Secada, & Wehlage, 1995). One section of the 
OTL included 5-point rating scales for four standards of the intellectual quality of instruction: 

• Construction of Knowledge: Higher Order Thinking. Instruction involves students in 
manipulating information about ideas by synthesizing, generalizing, explaining, 
hypothesizing, or arriving at conclusions that produce new meaning and understanding. 

• Disciplined Inquiry: Deep Knowledge. Instruction addresses central ideas of a topic or 
discipline with enough thoroughness to explore connections and relationships and to produce 
relatively complex understandings. 

• Disciplined Inquiry: Substantive Conversation. Students engage in extended conversational 
exchanges with the teacher or peers about subject matter in a way that builds an improved and 
shared understanding of ideas or topics. 

• Value Beyond School: Connections to the World Beyond the Classroom. Students make 
connections between knowledge and either public problems or personal experience (Newmann 
et al„ 1995). 

An aggregate score across three of the four standards was used as an overall measure of the 
Intellectual Challenge of instruction for each teacher. The score for Substantive Conversation was 
omitted from the composite score because ratings were highly influenced by the organizational 
structure of lessons. Specifically, lessons involving teacher-directed discussions typically yielded more 
public conversations, and thus, better opportunities to gather evidence on conversational exchanges 
than lessons with students working in small groups or individually. Additionally, to enhance observer 
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agreement for OTL ratings, we conducted training sessions for researchers immediately before each 
series of site visits began. We also utilized Many-Facet Rasch Measurement (Linacre, 2004) to adjust 
the measure of Intellectual Challenge for the relative severity (or leniency) of each observer during 
analyses. 

Table 4.5 reports the adjusted composite Intellectual Challenge scores for immersion and control 
teachers across four data-collection periods. When researchers conducted baseline observations in fall 

2004, sixth-grade control teachers’ mean Intellectual Challenge score (1.88) was significantly higher 
than immersion teachers’ instructional score (1.62). The difference represented a moderate effect size 
(ES = -0.33) favoring control teachers. Thus, control teachers in fall engaged students in lessons that 
required a higher level of thinking, delved into topics more thoroughly, and made stronger connections 
with students’ background experiences and the world beyond the classroom. On the contrary, in spring 

2005, sixth-grade teachers’ lessons at immersion schools received a slightly higher mean Intellectual 
Challenge score (1.87) than control teachers’ instruction (1.81). The difference between the groups, 
however, was statistically insignificant. 


Table 4.5. Adjusted Intellectual Challenge Scores for Immersion and Control Teachers 


Group 

Immersion 

Control 

1- value 

P 

Effect 

Size 

N 

Mean 

SD 

N 

Mean 

SD 

Fall 2004 

60 

1.62 

0.71 

65 

1.88 

0.87 

-1.84 

0.07-n 

-0.33 

Spring 2005 

106 

1.87 

0.93 

101 

1.81 

0.90 

0.48 

0.63 

0.07 

Spring 2006 

114 

1.82 

0.75 

103 

1.77 

0.76 

0.47 

0.64 

0.07 

Spring 2007 

95 

2.06 

0.80 

99 

1.91 

0.77 

1.28 

0.20 

0.19 


Notes. Observations at 21 immersion and 21 control schools. Intellectual Challenge of Instruction scores could range 
from 1 (low challenge) to 5 (high challenge). The rating for Substantive Conversation was deleted from the 
composite score. 'Difference is statistically significant at the 0.10 level. Effect size is Cohen's d. 


In spring 2006, lessons observed in sixth- and seventh-grade teachers’ classrooms at immersion 
schools received a slightly higher mean Intellectual Challenge score (1.82) than control teachers’ 
lessons (1.77), but not by a statistically significant margin. In spring 2007, lessons delivered by sixth-, 
seventh-, and eighth-grade teachers at immersion schools had a notably higher mean level of 
Intellectual Challenge (2.06) compared to control teachers’ instruction (1.91). Although the difference 
between groups was statistically insignificant, the small, positive effect size (0.19) showed that 
instruction at immersion schools was a bit more challenging. 

Table 4.6 su mm arizes findings across the data collection periods for each of the Intellectual Challenge 
domains. Effect sizes show that control teachers’ instruction in fall 2004, compared to immersion 
teachers, had a higher mean level of intellectual challenge for each of the four standards. However, in 
spring 2005, immersion teachers’ lessons received higher ratings for Higher Order Thinking (ES = 
0.18) and Depth of Knowledge (ES = 0.09). In spring 2006 and 2007, immersion teachers’ lessons, 
compared to control teachers, received higher Intellectual Challenge scores for each of the four 
standards. Effect sizes indicated that immersion teachers had a greater instructional emphasis on 
Higher Order Thinking (0.22 in 2006, 0.28 in 2007) and Connections beyond the Classroom (0.06 in 
2006, and 0.18 in 2007). 

In general, evidence accumulating over time suggests that the availability of laptop computers and 
digital resources has allowed students in technology immersion schools to experience more 
intellectually demanding work. Even so, the results across all observed classrooms indicate that 
lessons in middle-school core classes generally fail to intellectually challenge students, with average 
ratings mostly below 2 on the 5-point intellectual challenge of instruction scales. 
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Table 4.6. Adjusted Intellectual Challenge Scores for Immersion and Control Teachers, by 
Dimension and Year 


Standard 

Immersion 

Control 



Effect 

Mean 

SD 

Mean 

SD 

t-value 

P 

Size 

Fall 2004 

Higher Order Thinking 

1.67 

1.02 

1.80 

1.03 

-0.73 

0.470 

-0.13 

Depth of Knowledge 

1.60 

0.94 

1.85 

1.05 

-1.38 

0.171 

-0.25 

Substantive Conversation 

1.33 

0.77 

1.40 

0.75 

-0.49 

0.625 

-0.09 

Connections Beyond the Classroom 

1.35 

0.66 

1.48 

0.83 

-0.94 

0.349 

-0.17 

Spring 2005 

Higher Order Thinking 

1.89 

1.04 

1.71 

1.00 

1.21 

0.227 

0.18 

Depth of Knowledge 

1.83 

1.07 

1.73 

1.06 

0.65 

0.518 

0.09 

Substantive Conversation 

1.40 

0.74 

1.44 

0.84 

-0.36 

0.720 

-0.05 

Connections Beyond the Classroom 

1.79 

1.01 

1.82 

1.05 

-0.22 

0.827 

-0.03 

Spring 2006 

Higher Order Thinking 

1.91 

0.93 

1.71 

0.90 

1.64 

0.104 

0.22 

Depth of Knowledge 

1.85 

0.88 

1.83 

0.97 

0.13 

0.899 

0.02 

Substantive Conversation 

1.46 

0.73 

1.45 

0.92 

0.09 

0.932 

0.01 

Connections Beyond the Classroom 

1.63 

0.91 

1.58 

0.85 

0.41 

0.681 

0.06 

Spring 2007 

Higher Order Thinking 

2.20 

1.05 

1.92 

0.92 

1.98 

0.049* 

0.28 

Depth of Knowledge 

2.15 

0.98 

2.01 

0.98 

0.97 

0.332 

0.14 

Substantive Conversation 

1.46 

0.77 

1.38 

0.70 

0.75 

0.452 

0.11 

Connections Beyond the Classroom 

1.75 

0.97 

1.58 

0.88 

1.29 

0.198 

0.18 


Note. Rating scales developed by Newmann, Secada, & Wehlage (1995) ranged from 1 to 5. Teacher counts: fall 2004 (60 
immersion and 65 control), spring 2005 (105 immersion and 101 control), spring 2006 (114 immersion and 103), spring 
2007 (95 immersion and 99 control). *Statistically significant difference. Effect size is Cohen’s cl. 


Conclusions 

Although the level of implementation was generally low at many technology immersion schools in the 
third year, and control teachers benefited from initial steps toward the implementation of the 
technology immersion model (teacher laptops, digital resources, and targeted professional 
development), we found that being in a technology immersion school positively affected teachers in a 
number of ways. Key findings are the following: 

• Immersion teachers grew in technology proficiency and in their use of technology for 
professional productivity at significantly faster rates than control teachers. 

• Immersion teachers expressed increasingly stronger ideological affiliations across time with 
classroom technology integration and learner-centered practices than control teachers. At the 
same time, immersion teachers reported generally low and stable resistance to technology. 

• Teachers in immersion schools collaborated more often with their peers on technology-related 
instructional and learning issues than control teachers, and students in immersion classrooms 
used technology applications more often for core-subject learning activities. 

• Across both treatment and control campuses, school poverty was negatively associated with 
teachers’ growth on several technology-related indicators. Most importantly, teachers in 
schools with above average levels of school poverty grew in technology proficiency at a 
significantly slower rate. 


44 




Accumulating evidence suggests that the availability of laptop computers and digital resources 
has allowed students in technology immersion schools to experience more intellectually 
demanding work. However, ratings of the Intellectual Challenge of classroom instruction 
indicate that the intellectual demand of core-subject lessons was typically low across all 
middle-school classrooms. 




5. Effects of Technology Immersion on Students and Learning 


In the theoretical model for technology immersion, we assume that an improved school environment 
for technology will lead to teachers who have greater technology proficiency and use technology more 
often for their own professional productivity. Moreover, in a fully immersed school, teachers have 
students use technology almost daily in their classrooms, and technology provides a means to enhance 
the intellectual challenge and relevance of lessons. We also reason that improved school and 
classroom environments for technology will lead students to greater technology proficiency and use, 
more frequent peer collaboration, opportunities for more challenging and relevant school work, 
stronger engagement in school and learning, and enhanced personal self-direction. 

Consistent with our suppositions, findings reported in Chapter 4 confirm that teachers at immersion 
schools have grown individually in important areas. Immersion teachers, in comparison to their 
control counterparts, are more technically proficient and express a stronger ideological affiliation with 
immersion practices, and both teachers and their students use technology more often. Considering 
school and classroom conditions, we investigate in this section the effects of immersion on students 
and their learning experiences. 


Immersion Effects on Student Mediating Variables 

Data on student mediating variables come from paper- and-pencil surveys {Student Questionnaire and 
Style of Learning Inventory ) completed by three cohorts of students as baseline measures in fall and 
again as post-measures in spring of each project year. The Student Questionnaire measures students’ 
technology proficiency, technology use, and views on technical problems. The questionnaire also 
gauges students’ opportunities to work with peers in small groups and their satisfaction with school. 
The Style of Learning Inventory ( SLI) measures various aspects of students’ self-directed learning. 
Overall, response rates for the Student Questionnaire were in the 80% to 90% range across time 
periods, with only slight differences in response rates between cohorts and comparison groups. 
Response rates for the SLI ranged from 71% to 89% across administrations. With the exception of 
spring 2005, there were only slight differences in SLI response rates between cohorts and comparison 
groups. (See additional detail in the methodology chapter.) 

Sections to follow present findings for the three student cohorts. Since Cohorts 1 and 2 students 
completed surveys at three or more time points, we used hierarchical linear modeling (HLM) growth 
models to examine the effects of technology immersion on students’ individual growth rates for 
various measures. Cohort 3 students completed surveys at two time points, so we use HLM to estimate 
the effects of technology immersion on students’ spring scale scores. For all student groups, 
immersion effects are estimated for the following scales: Classroom Activities, Small-Group Work, 
Technical Problems, Technology Proficiency, Self-Directed Learning, and School Satisfaction. 
Cronbach’s alpha coefficients (measures of internal consistency reliability) for student-level scales 
ranged from 0.77 to 0.94 (see Appendix C for details). 
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HLM Analyses 

HLM Growth Analysis for Cohorts 1 and 2 Students 

Longitudinal data allowed researchers to examine the nature of student change over time. For 
Cohort 1. we collected data at four time points: fall 2004 (baseline) and spring 2005, 2006, and 2007 
(after the first, second, and third implementation years, respectively). For Cohort 2, we collected data 
at three time points: fall 2005 (baseline) and spring 2006 and 2007 (after the first and second 
implementation years). Analyses contrast the growth trajectories for students at immersion and control 
schools. We analyzed immersion effects on students’ self -perceptions and technology-related activities 
using three-level hierarchical linear growth models. These FILM models produce student- and school- 
specific effects (i.e., the extent to which scale scores vary across time, students, and schools). 

Level 1: Repeated-measures model. Level 1 is a repeated-measures model (i.e., survey time within 
students) that enables us to capture key features of growth (e.g., initial status, rate of change). In the 
model, Yw is the survey scale score at year t for student / in school j, and Survey Time is the point at 
which students completed surveys (Cohort 1,0 = fall 2004, 1 = spring 2005, 2 = spring 2006, and 
3 = spring 2007; Cohort 2, 0 = fall 2005, 1 = spring 2006, and 2 = spring 2007). The key parameters in 
the model are Ttoij and nuj- The coefficient n 0 ij represents the “initial status” (that is, the estimated initial 
scale score), for student / in school j in fall, and 7T /i; is the annual growth rate (rate of change) for 
student i in school j. The e ni is the error term (within-student measurement error) assumed to be 
normally distributed with a mean of 0 and a constant variance. Thus, at level 1, the model is 

Y tij - Koij + K Uj (Survey Time) tij + e tij . 

Level 2: Student-level model. The Level 2 model (between-students model) allows us to determine 
differences between students in features of growth (e.g., initial status \noij\, rate of change |7T/, ; ]). In the 
student-level model, p 00j represents the mean initial status of a more advantaged student (advantaged = 
0, disadvantaged =1) within school/ and [i i 0j represents the mean rate of change for an advantaged 
student within school j. The coefficients J3 0 ij and fi / / ; represent the effects of student poverty on initial 
status and school year rate of change, respectively. The r 0ij and r (H] are residuals (i.e., random effects). 
At level 2, the model is 


n 0 ij - fiooj + [Joiji Disadvantaged)^ + r 0ij 
7Ziij=Pi0j + pnj(Disadvantaged)ij + r nj 


Level 3: School-level model. At the school level (level 3), we examine how students’ initial status 
(fiooj) and growth (fhoj) vary across schools as a function of school-level random effects (juooj and fiioj), 
as well as school conditions, including immersion status (an indicator variable with a value of 0 for a 
control school and a value of 1 for an immersion school) and school poverty (a continuous variable 
with percentages ranging from 31% to 100%, and with a grand mean of 69.8%). That is, we theorize 
that being in an immersion school is positively related to students’ growth on technology-related 
scores, after controlling for the poverty level of the school. Thus, we pose the following school-level 
model: 

fiooj = yooo + yooi( Immersion status)j + yooi( School Poverty ) ; + [looj 

Pioj = )’ioo + yioi(Immersion status)j + yw 2 (Schooi Poverty )j + fi 10 j. 

In the model, yooo is the overall mean initial status of an advantaged student at a control campus with 
an average level of school poverty, and y 10 o is the overall mean student growth rate (of an advantaged 
student at a control campus with an average level of school poverty). The coefficients y 0 oi and y m 
represent the direction and strength of association of immersion status on school-level initial status and 
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growth rate, respectively. In addition, y 0 o 2 and y/ 02 1 'eprcsent the effect of school poverty on school- 
level initial status and growth rate, respectively. Analyses for Cohort 1 involved a total of 2,804 
students who were continuously enrolled in schools since October 2004, with 1,337 at immersion 
schools and 1,467 at control schools. Analyses for Cohort 2 involved 3,266 students continuously 
enrolled since October 2005, with 1,595 at immersion schools and 1,671 at control schools. 

HLM Analysis for Cohort 3 Students 

We analyzed the effects of immersion on Cohort 3 students’ scale scores for mediating variables using 
two-level hierarchical linear models. 

Level 1: Student-level model. In the student-level model, spring 2007 scale scores from surveys were 
regressed on fall 2006 scale scores, economic status (0 if not disadvantaged, 1 if disadvantaged), 
African American status (0 if not African American, 1 if African American), Hispanic status (0 if not 
Hispanic, 1 if Hispanic) and gender (0 if male, 1 if female). That is, 

Yy = p 0j + fi/Fall 2005 scale score) + ^/Disadvantaged) + [3 3 / African American) + 
ffji Hispanic) + f 5 /Female) + r,y 

Level 2: School-level model. A school-level model was developed to answer the question of whether 
immersion schools had higher scale scores than control schools, after controlling for initial scale 
scores, economic status, ethnicity, gender, and school-level poverty. That is, 

poj = )’oo + Jo/ Immersion dummy) + y 0 2 ( School Poverty) + ju 0 j. 

Immersion was an indicator variable with a value of 0 for a control school and 1 for an immersion 
school. School poverty was a continuous variable with percentages ranging from 31% to 100%, and a 
grand mean of 69.8%. For two-level HLM models, we calculated effect sizes (ES) in standard 
deviation units (Cohen’s d). Effect sizes greater than 0.5 are typically interpreted as large, 0.5 to 0.3 as 
moderate, 0.3 to 0.1 as small, and less than 0.1 as trivial. Analyses for Cohort 3 involved a total of 
3,993 students who were continuously enrolled in schools since October 2006, with 1,829 at 
immersion schools and 2,164 at control schools. 

Immersion Effects on Technology Experiences and Self-Perceptions 
Cohorts 1 and 2 

Analyses for Cohorts 1 and 2 involved the estimation of six, three-level HLM growth models. As 
Table 5.1 shows, we used separate models to estimate the effects of technology immersion on growth 
rates for measures of students’ school technology experiences, including Classroom Activities, Small- 
Group Work, and Technical Problems, as well as students’ self-perceptions of their Technology 
Proficiency, Self-Directed Learning, and School Satisfaction. Summary results show that technology 
immersion had positive effects on students in a number of areas. After controls for school poverty 
(percentage of economically disadvantaged students) and student economic disadvantage 
(qualification for free- or reduced-price lunch), estimated mean yearly rates of change for advantaged 
and disadvantaged immersion students revealed positive growth Lends favoring immersion students 
for Classroom Activities, Small-Group Work, and Technology Proficiency. Growth rates also showed 
that immersion students, compared to control, reported mounting Technical Problems using computers 
over time, with the growth-rate difference between groups statistically significant for Cohort 1 eighth 
graders. 
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The technology immersion model also assumes that having daily access to and personal responsibility 
for laptop computers will allow immersion students to become more Self-Directed Learners and will 
increase their School Satisfaction. Contrary to expectations, we found that as students in both the 
treatment and control groups advanced from sixth to eighth grade, they reported being less self- 
directed learners and expressed less satisfaction with school. There were no statistically significant 
differences between the views of immersion and control-group students. 


Table 5.1. Cohorts 1 and 2: Immersion Effects on Estimated Mean Growth Rates 
for Student Mediating Variables 


Scale Scores 

Immersion 
Effect Net 
of Student 
and School 
Poverty 

Immersion 
Yearly Growth Rate 

Control 

Yearly Growth Rate 

Advantaged 

Students 

Dis- 

advantaged 

Students 

Advantaged 

Students 

Dis- 

advantaged 

Students 

Cohort 1 (8th Graders) 

School Technology 



Classroom Activities (5-pt) 

Y pc 5 ! 1 ^ 

0.25 

0.30 

0.04 

0.08 

Small-Group Work (5-pt) 

Yes** 

0.02 

0.07 

-0.06 

0.00 

Technical Problems (5-pt) 

Yes* 

0.15 

0.17 

0.10 

0.13 

Student Self-Perceptions 



Technology Proficiency (5-pt) 

Yes 

0.31 

0.38*** 

0.27 

0.28 

Self-Directed Learning (7-pt) 

No 

-0.07 

-0.09 

-0.05 

-0.07 

School Satisfaction (5-pt) 

No 

-0.06 

-0.03 

-0.06 

-0.02 

Cohort 2 (7th Graders) 

School Technology 



Classroom Activities (5-pt) 

Y pc^ ^ 

0.30 

0.34 

0.06 

0.10 

Small-Group Work (5-pt) 

Yes** 

0.14 

0.13 

-0.01 

-0.02 

Technical Problems (5-pt) 

No 

0.24 

0.24 

0.17 

0.17 

Student Self-Perceptions 



Technology Proficiency (5-pt) 

Y ^ ^ 

0.43 

0.43 

0.27 

0.27 

Self-Directed Learning (7-pt) 

No 

-0.11 

-0.11 

-0.14 

-0.13 

School Satisfaction (5-pt) 

No 

-0.10 

-0.07 

-0.08 

-0.05 


Source: Student surveys completed during the 2004-05, 2005-06, and 2006-07 school years. 
Note. *p < .05. **p < .01. ***p < .001. Items measured on either a 5-point or 7-point scale. 


Cohort 3 

Analyses for Cohort 3 involved the estimation of six, two-level HLM models. We used separate 
models to assess the effects of technology immersion on students’ spring 2007 scores for the measures 
of school technology and student self-perceptions. Summary results presented in Table 5.2, similar to 
findings for Cohorts 1 and 2, show that technology immersion had a statistically significantly effect on 
four of the six mediating variables. After adjusting for fall 2006 scale scores, student demographic 
characteristics (gender, ethnicity, economic disadvantage), and school poverty (percentage of 
economically disadvantaged students), technology immersion had a significantly positive effect, on 
sixth graders’ spring scale scores for Classroom Activities (ES = 0.79), Small-Group Work (ES = 
0.29), and Technology Proficiency (ES = 0.30). And, consistent with Cohort 1, Cohort 3 immersion 
students reported having technical problems with computers significantly more often than their 
control-group counterparts (ES = 0.22). Also like Cohorts 1 and 2, there were no statistically 
significant effects of immersion on students’ Self-Directed Learning or School Satisfaction. 
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Table 5.2. Cohort 3 (Sixth Graders): Immersion Effects on Student Mediating Variables 


Scale 

Immersion Effect Net of 
Fall Score, Student 
Demographic 
Characteristics, & School 
Poverty 

Magnitude of 
Effect (d) in Standard 
Deviation Units 

School Technology 



Classroom Activities 

Y ge 5 ! 5 ^ ^ 

0.79 (large) 

Small-Group Work 

Y ^ ^ 

0.29 (small) 

Technical Problems 

Yes* 

0.22 (small) 

Student Self-Perceptions 



Technology Proficiency 

Y ge^ ^ ^ 

0.30 (small) 

Self-Directed Learning 

No 

0.11 (small) 

School Satisfaction 

No 

0.08 (trivial) 


Source: Student surveys completed in fall 2006 and spring 2007. 

Note. *p < .05. **p < .01. ***p < .001. Effect size is Cohen’s d value. The interpretation is that an effect 
size greater than 0.5 is large, 0.5-0. 3 is moderate, 0.3-0. 1 is small, and anything less than 0.1 is trivial. 


Sections to follow provide additional details for the HLM analyses. Student responses to specific 
scales help to explain outcomes for economically advantaged and disadvantaged students in 
technology immersion and control schools. 

School Technology 

Table 5.3 provides statistics for the HLM growth models estimating the immersion effects on 
Cohorts 1 and 2 students’ technology experiences. Specific scales are discussed below. 

Classroom Activities. Students reported the frequency with which their teachers had them use 
specific technology applications (e.g., use a word processor for writing, use a spreadsheet to calculate 
or graph, create a presentation) in their English language aits, mathematics, social studies, and science 
classes combined. Students reported their technology use on a 5-point scale ranging from 1 (never) to 
5 ( almost daily). As anticipated given the increased availability of hardware and software in 
immersion schools, treatment students had a significantly steeper growth rate for their frequency of 
technology use in core-subject classes. For Cohort 1 students, the yearly rates of change in Classroom 
Activities involving technology for economically advantaged and disadvantaged immersion students 
were 0.25 and 0.30 scale-score points, respectively. In contrast, advantaged and disadvantaged control 
students had relatively flat rates of change for classroom technology use (0.04 and 0.08 scale-score 
points, respectively). Similarly, the yearly rates of change in Classroom Activities for economically 
advantaged and disadvantaged Cohort 2 immersion students were 0.30 and 0.34 scale-score points, 
respectively, whereas advantaged and disadvantaged control students had relatively flat rates of 
change (0.06 and 0. 10 scale-score points, respectively). 

Figure 5.1 shows the estimated mean growth trajectories for the frequency of Classroom Activities for 
Cohort 1 students. Estimated mean scores for spring 2007 show that economically advantaged and 
disadvantaged students in immersion schools had Classroom Activities scores of 3.0 and 3.2, 
respectively, on the 5-point frequency scale, whereas mean scores for their control-group counterparts 
were 2.2 and 2.3, respectively. Despite significant increases in technology use by immersion students, 
mean use statistics indicated that students used various technology applications infrequently in 
classrooms (about once or twice a month). 
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Table 5.3. Cohorts 1 and 2: Immersion (Fixed) Effect Analyses of School Technology Variables 



Classroom Activities 
(with technology) 

Small-Group Work 

Technical Problems 

Dependent variable 
and predictor 

Gamma 

Coefficient f-value 

Gamma 

Coefficient f-value 

Gamma 

Coefficient f-value 

Cohort 1 (8th Graders) 

Initial status (fall 2004) 

2.020 

29 37*** 

2.802 

48.73*** 

2.366 

37.39*** 

Immersion 3 

0.257 

3.26** 

0.058 

0.99 

-0.138 

-1.95 

School Poverty 

-0.298 

-1.34 

-0.154 

-0.80 

-0.324 

-1.30 

Economic 

Disadvantage 

0.013 

0.43 

-0.051 

-0.96 

-0.071 

-1.04 

Growth rate 

0.040 

1.63 

-0.055 

-2.56* 

0.101 

3.27** 

Immersion 

0.212 

6.34*** 

0.072 

2.79** 

0.095 

2.43* 

School Poverty 

0.082 

0.86 

-0.052 

-0.59 

0.073 

0.53 

Economic 

Disadvantage 

0.044 

3.08** 

0.056 

2.79** 

0.024 

0.93 

Cohort 2 (7th Graders 

) 

Initial status (fall 2005) 

2.083 

32.57*** 

2.786 

45.50*** 

2.176 

39.28*** 

Immersion b 

0.145 

1.57 

-0.062 

-0.76 

-0.295 

-3 94 *** 

School Poverty 

0.454 

1.65 

0.144 

0.81 

0.264 

1.19 

Economic 

Disadvantage 

-0.013 

-0.34 

- 0.010 

-0.25 

-0.044 

-1.56 

Growth rate 

0.059 

1.40 

- 0.011 

-0.35 

0.168 

3.99** 

Immersion 

0.244 

4.16*** 

0.150 

3.62** 

0.072 

1.50 

School Poverty 

-0.174 

-1.05 

-0.023 

- 0.20 

-0.073 

-0.42 

Economic 

Disadvantage 

0.039 

1.62 

-0.013 

-0.61 

0.004 

0.17 


*p < .05. **p < .01. ***p < .001. 

■Immersion students had significantly higher initial classroom activities scores. A latent variable regression, 
controlling for the effect of this initial difference on the growth rate, indicated that the difference between the 
original (0.138) and adjusted (0.212) immersion coefficients was significant (the difference divided by the standard 
error of the difference equals -2.63). The growth rate coefficient adjusted for this difference is reported in the table. 
^Immersion students had significantly lower initial technical problems scores. A latent variable regression, 
controlling for the effect of this initial difference on the growth rate, indicated that the difference between the 
original (0.254) and adjusted (0.072) immersion coefficients was significant (the difference divided by the standard 
error of the difference equals 3.32). The growth rate coefficient adjusted for this difference is reported in the table. 
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Figure 5.1. Estimated mean growth trajectories for the frequency of Classroom Activities for 
Cohort 1 students, by economically advantaged and disadvantaged student groups in immersion 
and control schools. Statistics are displayed for disadvantaged students. 

Small-Group Work. Research studies consistently link one-to-one technology with a more 
collaborative classroom environment. Thus, our survey asked students to rate the frequency of their 
small-group interactions with classmates. Students rated statements, such as “we tutor or coach each 
other,” “brainstorm solutions to problems,” and “discuss assignments” on a 5-point scale ranging from 
1 {never) to 5 {almost daily). Growth rate coefficients show that students in immersion schools 
reported increasing opportunities for small-group work with their peers. Across cohorts, economically 
advantaged and disadvantaged immersion students had significantly positive yearly growth rates (0.02 
and 0.07 scale-score points, respectively, for Cohort 1; 0.14 and 0.13 scale-score points, respectively, 
for Cohort 2). Quite the opposite, students at control campuses reported less frequent small-group 
activities across survey times (yearly growth rates for advantaged and disadvantaged students ranged 
from 0 to -0.06 scale-score points). 

Technical Problems. Given the increased availability of technology in immersion schools and 
classrooms, we reasoned that students might encounter more technical problems. Thus, we asked 
students to indicate on a 5-point scale about how often various Technical Problems happened when 
they tried to use a computer at school. Across Cohorts 1 and 2, growth rates showed that immersion 
students increasingly reported technical problems using computers compared to control students, and 
for Cohort 1 students, the growth-rate difference was statistically significant. Figure 5.2 shows that 
Cohort 1 immersion students initially reported fewer technical problems than control students, but by 
the end of eighth grade, both economically advantaged and disadvantaged immersion students 
reported more technical troubles. Still, mean scores in spring 2007 indicated that eighth graders, on 
average, rarely (a few times a year) or sometimes (once or twice a month) had problems using 
computers at school. 
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Figure 5.2. Estimated mean growth trajectories for the frequency of Technical Problems for 
Cohort 1 students, by economically advantaged and disadvantaged student groups in immersion 
and control schools. 

Findings in Table 5.4 for Cohort 3 students generally mirror results for Cohorts 1 and 2. After 
adjusting for fall 2006 scale scores, student demographic characteristics (gender, ethnicity, economic 
disadvantage), and school poverty (percentage of economically disadvantaged students), technology 
immersion had a statistically significant effect on students’ Classroom Activities (0.61 scale-score 
point) and Small-Group Work (0.31 scale-score point). Cohort 3 sixth graders at immersion schools 
also reported significantly more frequent Technical Problems than control-group students (0.21 scale- 
score point), but problems using computers occurred rarely (a few times a year). 


Table 5.4. Cohort 3 (Sixth Graders): Immersion (Fixed) Effect Analyses of School 
Technology Variables 



Classroom Activities 
(with technology) 

Small-Group Work 

Technical Problems 

Gamma 

Coefficient 

f-value 

Gamma 

Coefficient 

f-value 

Gamma 

Coefficient 

f-value 

Intercept 

1.998 

35.66*** 

2.542 

46.54*** 

2.243 

3L22*** 

Immersion 

0.610 

8.51*** 

0.305 

4.19*** 

0.209 

2.65* 

School poverty 

-0.174 

-0.76 

0.060 

0.28 

0.058 

0.23 

Female 

0.004 

0.18 

0.076 

3.50** 

0.034 

1.17 

Hispanic 

0.046 

1.57 

0.044 

1.30 

0.008 

0.15 

African American 

0.138 

2.38* 

0.207 

4.88*** 

0.007 

0.11 

Disadvantaged 

0.112 

3.66** 

0.059 

1.44 

-0.010 

-0.29 

Fall 2006 score 

0.257 

10.28*** 

0.254 

15.65*** 

0.224 

8.48*** 


*p < .05. **p < .01. ***p < .001. 
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Student Self-Perceptions 

Table 5.5 provides statistical details for the HLM growth models related to Cohorts 1 and 2 students’ 
self-perceptions of their Technology Proficiency, Self-Directed Learning, and School Satisfaction. 
Individual scales are discussed below. 

Table 5.5. Cohorts 1 and 2: Immersion (Fixed) Effect Analyses for Student Self-Perception Variables 



Technology Proficiency 

Self-Directed Learning d 

School Satisfaction 

Gamma 

Coefficient 

f-value 

Gamma 

Coefficient 

f-value 

Gamma 

Coefficient 

f-value 

Cohort 1 (8th Graders) 

Initial status (fall 2004) 

2.978 

49.45*** 

4.552 

g9 97 *** 

3.740 

94.85*** 

Immersion'* 

0.043 

0.56 

0.106 

2 . 12 * 

0.064 

1.66 

School Poverty 

-0.126 

-0.54 

0.339 

2 . 22 * 

-0.131 

- 0.20 

Economic Disadvantage 

-0.341 

-8.75*** 

-0.051 

- 1.11 

-0.092 

-2.85** 

Growth rate 

0.267 

18.19*** 

-0.049 

-3.60** 

-0.055 

-5 4t*** 

Immersion 

0.043 

1.53 

-0.017 

- 1.01 

-0.008 

-1.80 

School Poverty 

0.005 

0.06 

-0.023 

-0.55 

0.163 

3.23** 

Economic Disadvantage 

0.008 

0.64 

-0.023 

- 1.68 

0.034 

1.29 

Economic x Immersion 

0.058 

4 22*** 

— 

— 

— 

— 

Cohort 2 (7th Graders) 

Initial status (fall 2005) 

2.993 

49.17*** 

4.787 

90.41*** 

3.865 

111.65*** 

Immersion 

0.009 

0.11 

-0.070 

- 1.10 

0.034 

0.79 

School Poverty 

0.215 

0.93 

0.303 

1.62 

-0.063 

-0.65 

Economic Disadvantage 

-0.290 

-6.75*** 

-0.129 

-3.57** 

-0.138 

-5.90*** 

Growth rate 

0.268 

g 23*** 

-0.137 

_7 n*** 

-0.075 

-3.66** 

Immersion 

0.160 

4 25*** 

0.024 

0.87 

- 0.021 

-0.78 

School Poverty 

-0.149 

- 1.20 

-0.026 

-0.32 

0.077 

1.34 

Economic Disadvantage 

0.006 

0.20 

0.004 

0.22 

0.026 

1.55 


*p < .05. **p < .01. ***p < .001. 

d Immersion students had significantly higher initial self-directed learning scores. A latent variable regression, 
controlling for the effect of this initial difference on the growth rate, indicated that the difference between the 
original and adjusted immersion coefficients was not significant (the difference divided by the standard error of the 
difference = 0 . 88 ). 

Technology Proficiency. As a measure of their Technology Proficiency, students rated their skills in 
using technology applications on a 5-point scale ranging from 1 (I can do this not at all or barely) to 5 
(/ can do this extremely well). Students indicated their skill level on statements aligned with the Texas 
Technology Applications Standards. Over the first two project years, Cohort 1 immersion students 
rated their Technology Proficiency as significantly more advanced than control-group students. By the 
end of eighth grade, however, a different trend emerged. A significant interaction was detected 
between technology immersion and students’ economic status. While there was no main immersion 
effect on Cohort 1 students’ growth in Technology Proficiency, economically disadvantaged students 
in immersion schools grew in proficiency at a significantly faster rate (0.38 scale-score point per year) 
compared to their more affluent immersion peers (0.31 scale-score point). Thus, as Figure 5.3 
illustrates, by spring 2007, economically disadvantaged immersion students (M = 3.8) narrowed the 
proficiency gap with advantaged immersion students (M = 4.0), closed the proficiency gap with 
advantaged control students (M = 3.8), and exceeded economically disadvantaged control students in 
proficiency (M = 3.5). Thus, technology immersion had a positive and enduring effect on the 
technology proficiencies of economically disadvantaged students. 
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Figure 5.3. Significant cross-level interaction between immersion and economically 
disadvantaged status for Cohort 1 Students’ Technology Proficiency scores (in schools with 
average poverty). 

Results for Cohort 2 replicated results from previous project years. Both economically advantaged and 
disadvantaged immersion students grew in Technology Proficiency at a significantly faster rate than 
their counterparts in control schools. The yearly rates of change in proficiency for economically 
advantaged and disadvantaged students, respectively, in immersion schools were 0.43 and 0.43 scale- 
score points compared with 0.27 and 0.27 scale-score points for advantaged and disadvantaged control 
students. Thus, economically disadvantaged students in immersion schools surpassed advantaged 
control students in proficiency by the end of seventh grade (estimated mean scores of 3.6 and 3.5, 
respectively). 

Self-Directed Learning. Self-direction, as measured by the SLI for this study, includes statements 
relative to students’ forethought (e.g., goal setting, strategic planning, self-efficacy beliefs, intrinsic 
effort), performance/volition control (e.g., attention focusing, self-monitoring, and help seeking), and 
self-reflection (e.g., self-evaluation, adaptivity). Although prior research suggests that the 
individualized learning opportunities allowed through one-to-one technology will positively affect 
students’ self-regulated learning, our results revealed no significant immersion effects on either 
Cohorts 1 or 2 students’ growth in self-direction. As both immersion and control students progressed 
through seventh and eighth grade, their responses to statements revealed significantly negative growth 
trends. For Cohort 1, the estimated yearly rates of change in self-direction for advantaged and 
disadvantaged students in immersion schools were -0.07 and -0.09 scale-score points, respectively, 
compared to -0.05 and -0.07 scale-score points, respectively, for their control -group counterparts. 
Correspondingly, for Cohort 2, the estimated yearly rates of change in self-direction for advantaged 
and disadvantaged students were similarly negative for immersion students (-0. 1 1 and -0.11, 
respectively) and control students (-0.14 and -0.13). Overall findings indicated that neither seventh nor 
eighth graders considered themselves to be strongly self-directed learners. 
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School Satisfaction. Students also rated their level of School Satisfaction by indicating the extent of 
their agreement with statements on a 5-point scale ranging from 1 ( strongly disagree ) to 5 ( strongly 
agree). For example, students responded to items measuring their satisfaction with class work, the 
meaningfulness of class work, and the extent to which they perceived their class work to be useful to 
them in the future. As sixth graders, both immersion and control students generally agreed with 
statements measuring their school satisfaction. Flowever, both treatment- and control-group students 
reported lower levels of school satisfaction across time. The estimated yearly rates of change in 
satisfaction for Cohorts 1 and 2 immersion students ranged from -0.03 to -0.10 scale-score points. 
Similarly, control students expressed declining levels of satisfaction (-0.02 to -0.08 scale-score points 
per year). 

Findings for Cohort 3 students are consistent with results for Cohorts 1 and 2. After adjusting for fall 
2006 scale scores, student demographic characteristics, and school, technology immersion had a 
significantly positive effect on students’ 2006 scale scores for Technology Proficiency. On the other 
hand, there were no significant effects of immersion on sixth graders’ Self-Directed Learning or 
School Satisfaction. Similar to previous years, females rated their technology proficiency as higher 
than males, they perceived themselves to be more self-directed learners, and they expressed greater 
satisfaction with the kind of academic work they do in middle schools. 


Table 5.6. Cohort 3 (Sixth Graders): Immersion (Fixed) Effect Analyses of Student 
Self-Perception Variables 



Technology Proficiency 

Self-Directed Learning 

School Satisfaction 

Gamma 

Coefficient 

f-value 

Gamma 

Coefficient 

f-value 

Gamma 

Coefficient 

f-value 

Intercept 

3.144 

62.11*** 

4.468 

99.06*** 

3.625 

77 67*** 

Immersion 

0.330 

6.56*** 

0.056 

1.32 

0.068 

1.75 

School poverty 

-0.345 

-2.48* 

0.067 

0.51 

0.075 

0.48 

Female 

0.085 

3.09** 

0.073 

3.22** 

0.120 

j 27*** 

Hispanic 

-0.012 

-0.37 

0.017 

0.38 

0.026 

0.57 

African American 

-0.024 

-0.60 

0.094 

2.16* 

-0.017 

-0.29 

Disadvantaged 

-0.073 

-1.66 

-0.036 

-1.90 

-0.038 

-1.09 

Fall 2006 score 

0.526 

23.36*** 

0.571 

25.39*** 

0.396 

12.49*** 


*p < .05. **p < .01. ***p < .001. 


Immersion Effects on Student Engagement 

We also theorized that greater technology access and use would lead to improvements in student 
conduct, and consequently, fewer discipline problems as well as increased school attendance. Findings 
on student engagement presented below show that immersion had positive effects on student discipline 
and behavior, but negative effects on school attendance. 

Student Discipline and Behavior 

As one measure of engagement, we collected student-level data from schools on disciplinary actions 
occurring during the 2006-07 school year. Texas requires that schools report each disciplinary action 
that results in a removal of a student from their regular academic program for a full school day. 
Therefore, we compared the frequency of the disciplinary occurrences at treatment and control schools 
for Cohorts 1, 2, and 3 students. Preliminary statistical tests showed generally non-normal and 
negatively skewed distributions of disciplinary actions for each cohort (see Kolmogorov-Smirnov tests 
for normality in Table 5.7). Flowever, because t-tests of differences between mean scores are robust to 
violations of the normality assumption (Rasch & Guiard, 2004), we used this parametric procedure to 
test for differences between groups. Results for independent Mcsts show statistically significant 
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differences between the frequency of disciplinary actions at immersion and control schools, favoring 
immersion across three cohorts (Table 5.7). Figure 5.4 compares the number of students and average 
number of disciplinary actions for immersion and control schools for each of the student cohorts. 

Table 5.7. Statistics for Comparisons of Disciplinary Actions at Immersion and Control Schools 


Cohort 

Kolmogorov- 
Smirnov z 
Test for 
Normality 

Mann-Whitney 
Rank-Sum U 
Test z statistic 

Independent 

Samples 

r-test 

Effect Size 

Cohort 1 (8th Graders) 

1.27 

3.18** 

4 Q9*** 

- 0.11 

Cohort 2 (7th Graders) 

1.91** 

4.88*** 

5.83*** 

-0.16 

Cohort 3 (6th Graders) 

1.43* 

3.80*** 

4 93 *** 

-0.13 


Note. *p < .05. **p < .01. ***p < .001 . Two outliers were removed from the analyses: a Cohort 1 control 
student with 112 disciplinary actions and a Cohort 3 immersion student with 1 1 1 disciplinary actions. 
Removing the outliers did not affect the conclusions. 



Cohort 1 , Eighth Cohort 2, Seventh Cohort 3, Sixth Graders 
Graders Graders 


□ Immersion □ Control 


Figure 5.4. Number of students disciplined and average number of disciplinary actions for 
Cohorts 1, 2, and 3 students who attended technology immersion and control schools. 

First, Cohort 1 eighth graders at immersion schools had significantly fewer disciplinary actions than 
control students (t = 4.01, p < 0.001). Specifically, 2,899 control-group students had an average of 
0.90 disciplinary actions compared to 2,584 immersion students who had an average of 0.65 
disciplinary events. Similarly, Cohort 2 seventh graders at immersion schools had significantly fewer 
disciplinary actions than students at control schools ( t - 5.83, p < 0.001). Specifically, 2,899 control- 
group students had an average of 0.86 disciplinary actions compared to 2,624 immersion students who 
had an average of 0.53 disciplinary actions. Likewise, Cohort 3 students had significantly fewer 
disciplinary events (t = 4.93, p < 0.001). For Cohort 3, 2,846 sixth graders at control schools had an 
average of 0.75 disciplinary actions compared to 2,573 sixth graders at immersion schools who had 
0.47 disciplinary actions, on average. Effect sizes for the mean differences between groups were small 
across cohorts (-0.11, -0.16, and -0.13, respectively). 

Also note that the more conservative non-parametric Mann-Whitney U test showed significantly fewer 
disciplinary actions at immersion campuses (Table 5.7). For example, for Cohort 1, the mean rank for 
the immersion students, 2,688.41 was significantly less than the mean rank for control students, 
2,789.77 (z = 3.18, p = 0.001). Similarly, for Cohorts 2 and 3, the mean ranks for immersion students 
were significantly less than the mean ranks for control students. 
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Overall, second-year findings on student discipline and behavior mirror results for the first and second 
project years. Evidence shows that sixth, seventh, and eighth graders attending technology immersion 
schools have fewer disciplinary referrals than their counterparts in control schools. Although the 
estimated size of differences between groups is considered statistically small, having fewer 
disciplinary actions per student in middle schools may have practically important benefits. 

Student Attendance 

School attendance rates (absolute values). Another indicator of engagement is students’ school 
attendance. Accordingly, we compared the annual attendance rates for Cohort 1 students for the year 
before project implementation and for three implementation years. Cohort 2 students for the year 
before implementation and for two implementation years, and Cohort 3 students for the year before 
implementation and after one implementation year (Table 5.7). 


Table 5.7. School Attendance Rates for Cohorts 1, 2, and 3 Students 



Immersion 

Control 



Mean 

SD 

Mean 

SD 

Difference 

Cohort 1 (8th) 






2003-04 

97.23 

3.86 

97.54 

3.05 

-0.31 

2004-05 

96.78 

3.69 

97.03 

3.28 

-0.25 

2005-06 

96.07 

4.66 

96.84 

3.53 

-0.77 

2006-07 

95.43 

5.51 

96.53 

4.39 

-1.10 

Cohort 2 (7th) 






2004-05 

97.02 

3.88 

97.32 

3.16 

-0.30 

2005-06 

96.42 

4.20 

96.91 

3.61 

-0.49 

2006-07 

95.68 

5.24 

96.46 

4.16 

-0.78 

Cohort 3 (6th) 






2005-06 

96.90 

4.59 

97.16 

4.21 

-0.26 

2006-07 

96.05 

5.33 

96.77 

4.77 

-0.72 


Results for Cohort 1 students show that the average attendance rate of students in immersion schools 
was approximately 0.3 percentage point lower than the attendance rate of control students in the first 
year, and the attendance-rate gap increased incrementally to about 1.1 percentage points lower after 
three implementation years. Similarly, for Cohort 2, the average attendance rate of immersion students 
was about 0.3 percentage point lower than the attendance rate of control students in the year before 
implementation, and after two implementation years, the attendance-rate differential was about 0.8 
percentage point lower. In the same way, the average attendance rate of Cohort 3 immersion students 
was about 0.3 percentage point lower than the control group prior to project implementation, and one 
year after implementation, the attendance rate of immersion students was 0.7 percentage point lower. 

H E M analyses of attendance. To test the effects of immersion on student attendance, while 
controlling for school and student characteristics, we conducted HLM analyses. For Cohorts 1 and 2, 
we used three-level HLM growth models to examine changes in school attendance rates over time. For 
Cohort 3 students, we used a two-level HLM model to examine the effects of immersion on students’ 
2006-07 attendance rate. Table 5.8 presents the HLM statistics for each of the student cohorts. 
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Table 5.8. Cohorts 1, 2, and 3: Immersion (Fixed) Effect Analyses of Student Attendance 


Group 

School-Level 

Analysis 

Gamma 

Coefficient 

t 

Cohort 1 (8th Graders) 




3-Level HLM Growth Model 

Initial attendance (2004) 

97.706 

567.25*** 


Immersion 

-0.045 

-0.19 


School poverty 

2.230 

3.56** 


Disadvantaged 

-0.651 

-3 79 *** 


Growth rate 

-0.165 

-2.72* 


Immersion 

-0.272 

-3.04** 


School poverty 

-0.260 

-1.05 


Disadvantaged 

-0.119 

-2.03* 

Cohort 2 (7th Graders) 




3-Level HLM Model 

Initial attendance (2005) 

97.518 

671.67*** 


Immersion 

-0.262 

-1.41 


School poverty 

1.550 

2.71* 


Disadvantaged 

-0.487 

-3.59** 


Growth rate 

- 0.120 

-1.38 


Immersion 

-0.249 

-2.28* 


School poverty 

-0.437 

-1.59 


Disadvantaged 

-0.309 

-4.26*** 

Cohort 3 (6th Graders) 




2-Level HLM Model 

Intercept 

96.886 

482.35*** 


Immersion 3 

-0.564 

-2.67* 


School poverty 

0.393 

0.66 


Prior attendance 

0.204 

1.96* 


Disadvantaged 

0.382 

1.58 


Female 

0.262 

1.53 


Hispanic 

-0.543 

_4 74 *** 


African American 

0.622 

17.00*** 


*p < .05. **p < .01. ***p < .001. 


Contrary to expectations, results indicate that technology immersion had a significantly negative effect 
on students’ school attendance. For Cohort 1 (eighth graders) estimated immersion effects on schools’ 
adjusted average rates of attendance (controlling for student and school poverty) show that average 
attendance rates for economically advantaged immersion- and control-group students in schools with 
average rates of school poverty decreased as students advanced from fifth to eighth grade (see Figure 
5.5). The yearly estimated negative rate of change in attendance for immersion students (-0.44 
percentage point) was greater than the annual change for control students (-0.17 percentage point). 
Thus, at the end of eighth grade, advantaged students in immersion schools had an estimated average 
attendance rate of 96.3% percent compared to 97.2% for control students, with the statistically 
significant difference favoring control students. Attendance rates for economically disadvantaged 
students decreased at an even faster pace, with yearly negative change rates for disadvantaged students 
in immersion schools greater than the declining rates for control students (-0.56 percentage point 
versus -0.28 point, respectively). 
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Figure 5.5. Estimated attendance rates for Cohort 1 economically advantaged and 
disadvantaged students in immersion and control schools with average rates of school poverty. 

Similar to Cohort 1 . average school attendance rates for economically advantaged Cohort 2 immersion 
and control-group students in schools with average rates of school poverty decreased as students 
advanced from fifth to seventh grade. The yearly estimated negative rate of change in attendance for 
immersion students (-0.37 percentage point) was greater than the annual change for control students 
(-0.12 percentage point). Thus, at the end of seventh grade, advantaged students in immersion schools 
had an estimated average attendance rate of 96.5% percent compared to 97.3% for control students, 
with the significant difference favoring control students. Attendance rates for economically 
disadvantaged students decreased at a faster pace, with yearly negative change rates for disadvantaged 
students in immersion schools greater than the rates for control students (-0.68 percentage point versus 
-0.43 point, respectively). Thus, by the end of seventh grade, economically disadvantaged students in 
immersion schools had an attendance rate of 95.4% compared to 96.2% for control students. HLM 
results for Cohort 3, similarly, showed that after controlling for school and student characteristics, 
immersion students attended school at a statistically lower rate than control students (-0.56 percentage 
point). 

Conclusions 

In the third project year, we investigated the effects of technology immersion on students and learning 
for Cohort 1 (eighth graders), Cohort 2 (seventh graders), and Cohort 3 (sixth graders). After 
controlling for important school and student characteristics, data across three student cohorts confirm 
the hypothesized effects of immersion on some mediating valuables, whereas outcomes for other 
variables are contrary to expectations. Key findings include the following. 

• Across three cohorts, technology immersion positively affected students’ classroom 
technology use (Classroom Activities) and interactions with peers (Small-Group Work). 
Students in immersion schools used various technology applications significantly more often 
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in their core-subject classrooms than control students. They also had significantly more 
frequent opportunities to learn in small groups with their classmates. 

Across three cohorts, technology immersion positively affected students’ Technology 
Proficiency, although some distinctions among Cohort 1 students emerged in the third year. 
Cohorts 2 and 3 immersion students grew in proficiency at a significantly faster rate or to a 
higher level than control students. For Cohort 1, economically disadvantaged immersion 
students grew in proficiency at a significantly faster rate than their more affluent peers. 
Certainly, technology immersion has substantially closed the technology equity gap for 
economically disadvantaged students. 

Cohorts 1 and 2 technology immersion students, compared to control-group students, reported 
mounting Technical Problems using computers over time, with the growth in problems 
statistically significant for Cohort 1 eighth graders. Similarly, Cohort 3 immersion students 
reported having technical problems using computers significantly more often than control 
students. 

Across three cohorts, technology immersion students, who had access to personal laptop 
computers and resources for learning, regarded themselves as no more Self-Directed Learners 
than control students. As both immersion and control students progressed from sixth to eighth 
grade they reported significantly less self-directed learning behaviors. Likewise, immersion 
and control-group students expressed similar levels of school satisfaction, with both groups 
reporting significantly lower levels of school satisfaction as they progressed to higher grade 
levels. 

Across three cohorts, technology immersion positively affected student discipline and 
behavior. Students in immersion schools, on average, had proportionately fewer behavioral 
and disciplinary problems that removed them from the regular academic program than their 
counterparts in control schools. 

Contrary to expectations, results indicated that technology immersion had a negative effect on 
students’ school attendance. Across three cohorts, estimated attendance rates for immersion 
students were significantly lower than attendance rates for control-group students. Thus, 
students in immersion schools attended school less regularly than control students, and their 
school attendance declined each implementation year. 



6. Effects of Technology Immersion on Student Achievement 


The Texas Technology Immersion project aims ultimately to increase middle school students’ 
achievement in core academic subjects (English language arts, mathematics, science, and social 
studies) as measured by the Texas Assessment of Knowledge and Skills (TAKS). Accordingly, we 
theorize that students in fully immersed schools will experience school and classroom conditions that 
promote more individualized learning, more intellectually challenging work, and enhanced 
engagement in school and learning. In turn, changes in students and their learning experiences will 
contribute to enhanced performance on state assessments. In the third year of the technology 
immersion project, as detailed in previous report chapters, we have noted teachers’ substantial growth 
across years in technology proficiency and the frequency of classroom technology use, as well as 
improvements in students’ technology proficiency and use, and their school behavior. The following 
sections present academic achievement results for Cohorts 1, 2, and 3 students who were enrolled 
continuously in the 21 immersion and 21 control schools through TAKS testing in April 2007. 

Texas Assessment of Knowledge and Skills 
Passing Standards and Scale Scores 

The TAKS is Texas’ criterion-referenced assessment that measures students’ mastery of the state’s 
content standards, the Texas Essential Knowledge and Skills (TEKS). At the middle school, TAKS 
assesses reading and mathematics at grades 6, 7, and 8, writing at grade 7, and science and social 
studies at grade 8. This study uses several types of TAKS scores. 

• Met the standard. This score represents satisfactory academic achievement. Students who 
meet this standard performed at a level that was at or somewhat above the state passing 
standard. Thus, students demonstrated a sufficient understanding of the knowledge and skills 
measured at the grade level. 

• Commended performance. This score represents high academic achievement. Students who 
meet this standard performed at a level that was considerably above the state passing standard. 
Therefore, students demonstrated a thorough understanding of the knowledge and skills 
measured at the grade level. 

• TAKS scale score. The scale score is a statistic that provides a comparison of scores with a 
standard set at 2100 for each grade level. The scale score can be used to determine whether a 
student met the minimum standard or achieved commended performance, but it cannot be 
used to evaluate a student’s progress across grades or subject areas. TAKS scale scores are 
used to calculate standardized scores for this study. 

Texas has phased-in increasingly rigorous passing standards on the TAKS. In 2004-05, passing 
standards recommended for reading, mathematics, writing, social studies, and grade 5 science by the 
State Board of Education panel were fully implemented. For the newer grade 8 science test, the panel- 
recommended standard must be met in 2007-08. For this study, all TAKS scores reported are based on 
panel-recommended standards. 
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Standard Scores 


In addition to the scores provided by the Texas Education Agency (TEA), researchers generated 
standard scores that are used to compare student progress on TAKS across grade levels. A 
standardized score — or z score — was calculated for each student and for every testing occasion and 
subject. The z score is calculated by subtracting the statewide mean grade-level scale score from each 
student’s scale score and dividing by the statewide scale score standard deviation. The z score, which 
has a mean of zero and a standard deviation of 1 .0, indicates how many standard deviations from the 
mean a score lies. 

One characteristic of z scores is that about half of the scores are negative, and negative scores may be 
difficult to fully understand. To overcome this limitation, we have transformed students’ z scores into 
normalized scores, or T scores. T scores are scores with a mean of 50 and a standard deviation of 10. 
Thus, a student who scores at the state average will have a TAKS T score of 50. A student who has a 
score of 60 will be one standard deviation above the state average, and a student who has a score of 40 
will be one standard deviation below the state average. 

Progress in Meeting TAKS Standards 


TAKS Reading 

One measure of student academic outcomes is their progress toward meeting TAKS passing and 
commended performance standards. Information in Table 6.1 compares the absolute performance of 
students in immersion and control schools for TAKS reading. 


Table 6.1. TAKS Passing and Commended Performance Rates for Reading 


TAKS Test 

Group 

N 

2004 

Percent 

2005 

Percent 

2006 

Percent 

2007 

Percent 

Baseline to 
2007 

Difference 

Met Standard 

Cohort 1 

Grade 5 to 8 

Immersion 

1,380 

68.4 

76.4 

73.6 

86.1 

17.7 

Control 

1,613 

73.8 

82.5 

78.4 

88.1 

14.3 

Cohort 2 

Grade 5 to 7 

Immersion 

1,546 

— 

67.1 

87.6 

80.9 

13.8 

Control 

1,725 

— 

73.7 

91.2 

84.1 

10.4 

Cohort 3 

Grade 5 to 6 

Immersion 

1,778 

— 

— 

74.0 

87.0 

13.0 

Control 

1,991 

— 

- 

80.0 

91.6 

11.6 

Commended 

Performance 

Cohort 1 

Grade 5 to 8 

Immersion 

1,380 

19.7 

29.3 

17.1 

35.1 

15.4 

Control 

1,613 

23.2 

35.6 

18.9 

40.8 

17.6 

Cohort 2 

Grade 5 to 7 

Immersion 

1,546 

— 

17.5 

29.4 

21.4 

3.9 

Control 

1,725 

— 

19.0 

33.8 

20.5 

1.5 

Cohort 3 

Grade 5 to 6 

Immersion 

1,778 

— 

— 

18.3 

39.5 

21.3 

Control 

1,991 

- 

— 

20.4 

47.1 

26.7 


Source: Analysis of individual student data from TEA. 

Note. The 2004 passing rates are based on 2005 standards. Students in 21 immersion and 21 control schools that had 
TAKS scores and attended the same school across years. Italic text denotes baseline scores. 


Results show that Cohorts 1, 2, and 3 students at immersion schools had slightly lower passing rates in 
spring 2007 for TAKS reading than students at control campuses. However, students at immersion 
campuses had greater baseline-to-2007 passing increases. For Cohort 1, TAKS-score comparisons 
between 2004 (5th grade baseline) and 2007 (8th grade) revealed slightly larger reading gains for the 
immersion group (17.7 percentage points versus 14.3 points for the control group). Similarly, for 
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Cohort 2, the TAKS passing rate difference between 2005 (5th grade baseline) and 2007 (7th grade) 
favored students at immersion schools (13.8 percentage points versus 10.4 points for control students). 
Likewise, passing rate comparisons between 2006 (5th grade baseline) and 2007 (6th grade) favored 
Cohort 3 students at immersion schools (13.0 percentage points versus 11.6 points for control 
students). 

For commended performance, Cohorts 1 and 3 students at control schools had higher 2007 
achievement rates and larger baseline-to-2007 gains than immersion students. Conversely, Cohort 2 
students at immersion schools had a slightly higher commended performance rate and showed larger 
gains. 

TAKS Mathematics 

Similar to reading, results for TAKS mathematics show that Cohorts 1, 2, and 3 students at immersion 
schools had slightly lower TAKS mathematics passing rates in spring 2007 than students at control 
campuses (Table 6.2). However, TAKS-score comparisons across years revealed slightly smaller 
TAKS mathematics decreases for each of the immersion groups. 


Table 6.2. TAKS Passing and Commended Performance Rates for Mathematics 


TAKS Test 

Group 

N 

2004 

Percent 

2005 

Percent 

2006 

Percent 

2007 

Percent 

Baseline to 
2007 

Difference 

Met Standard 

Cohort 1 

Immersion 

1,397 

70.7 

62.3 

65.7 

70.7 

0.0 

Grade 5 to 8 

Control 

1,616 

73.2 

68.4 

68.5 

71.5 

-1.7 

Cohort 2 

Immersion 

1,560 

— 

73.3 

72.4 

72.2 

-1.1 

Grade 5 to 7 

Control 

1,750 

— 

79.3 

74.9 

73.0 

-6.3 

Cohort 3 

Immersion 

1,782 

— 

— 

77.7 

72.5 

-5.2 

Grade 5 to 6 

Control 

2,031 

- 

- 

84.0 

77.2 

-6.8 

Commended 1 

Performance 

Cohort 1 

Immersion 

1,397 

24.2 

19.8 

11.0 

15.7 

-8.5 

Grade 5 to 8 

Control 

1,616 

24.6 

21.8 

9.7 

13.8 

-10.8 

Cohort 2 

Immersion 

1,560 

— 

22.6 

22.3 

15.9 

-6.7 

Grade 5 to 7 

Control 

1,750 

— 

25.3 

22.6 

12.0 

-13.3 

Cohort 3 

Immersion 

1,782 

— 

— 

31.5 

27.6 

-3.9 

Grade 5 to 6 

Control 

2,031 

- 

— 

36.4 

27.5 

-8.9 


Source: Analysis of individual student data from TEA. 

Note. The 2004 passing rates are based on 2005 standards. Students in 21 immersion and 21 control schools that had 
TAKS scores and attended the same school across years. Italic text denotes baseline scores. 


Overall, students had greater difficulty meeting commended standards for mathematics compared to 
reading. However, comparison-group Lends showed that immersion students had generally higher 
TAKS mathematics commended performance rates and smaller losses. Baseline-to-2007 differences 
for commended performance favored Cohort 1 eighth graders in immersion schools (-8.5 percentage 
points versus -10.8 points for control students). Cohort 2 seventh graders (-6.7 percentage points 
versus -13.3 points), and Cohort 3 sixth graders (-5.2 percentage points versus -6.8 points). 

TAKS Social Studies, Science, and Writing 

The TAKS reading and mathematics tests are administered annually, whereas TAKS tests are 
administered periodically in other subject areas. In the third year. Cohort 1 eighth graders completed 
TAKS social studies and science assessments, while Cohort 2 seventh graders completed TAKS 
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writing. Baseline measures were available for TAKS science in grade 5 and writing in grade 4. There 
was no pre-measure for TAKS social studies. 

Results for TAKS social studies in Table 6.3 show that Cohort 1 students at both immersion and 
control schools had si mi lar TAKS passing rates for social studies in 2007, but commended 
performance rates were slightly higher for students at immersion campuses (28.5% versus 26.7%). 
Conversely, Cohort 1 students at immersion schools had slightly lower TAKS passing rates for science 
in 2007 (65.5%) than control students (67.6%) but nearly identical baseline to 2007 passing rate 
decreases (-2.2 and -2.6 percentage points, respectively). Control students also achieved 2007 
commended performance in science at a slightly higher rate than immersion students 
and had a baseline to 2007 gain rather than a loss. 

Table 6.3. Cohort 1 (Eighth Graders in 2006-07): 

TAKS Passing and Commended Performance Rates for Social Studies and Science 


TAKS Test 

Group 

N 

2004 
Grade 5 
Percent 

2007 
Grade 8 
Percent 

Baseline to 
2007 

Difference 

Met Standard 

Social 

Immersion 

1,459 

— 

85.6 

— 

Studies 

Control 

1,710 

— 

86.0 

— 

Science 

Immersion 

1,367 

67.7 

65.5 

-2.2 


Control 

1,600 

70.2 

67.6 

-2.6 

Commended 1 

Performance 

Social 

Immersion 

1,459 

— 

28.5 

— 

Studies 

Control 

1,710 

— 

26.7 

— 

Science 

Immersion 

1,367 

14.0 

12.3 

-1.7 


Control 

1,600 

12.3 

14.0 

1.7 


Source: Analysis of individual student data from TEA. 

Note. The 2004 passing rates are based on 2005 standards. Students had TAKS scores 
and attended the same school across years. Italic text denotes baseline scores. 


Table 6.4 shows that Cohort 2 students at immersion schools had slightly lower TAKS passing rates 
for writing in 2007 than control students, and immersion students’ TAKS-score gains between 2004 
(4th grade baseline) and 2007 (7th grade) were slightly smaller. Control students also achieved 
commended performance in writing at a higher rate and had larger gains than immersion students 
(12.4 percentage points versus 10.7 points). 


Table 6.4. Cohort 2 (Seventh Graders in 2006-07): 

TAKS Passing and Commended Performance Rates for Writing 


TAKS 

Test 

Group 

N 

2004 
Grade 4 
Percent 

2007 
Grade 7 
Percent 

Baseline to 
2007 

Difference 

Met Standard 

Writing 

Immersion 

1,457 

92.2 

94.6 

2.4 


Control 

1,631 

92.8 

95.7 

2.9 

Commended Performance 

Writing 

Immersion 

1,457 

18.0 

28.7 

10.7 


Control 

1,631 

19.0 

31.4 

12.4 


Source: Analysis of individual student data from TEA. 

Note. The 2004 passing rates are based on 2005 standards. Students had TAKS scores 
and attended the same school across years. Italic text denotes baseline scores. 
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Altogether, TAKS passing rates provide important evidence that helps to understand student progress 
toward meeting state standards — however, additional statistical analyses are necessary to assess the 
effects of immersion on student achievement. 

Effects of Immersion on Academic Achievement 

Researchers used hierarchical linear modeling (HLM) to estimate the effects of immersion on 
students’ academic achievement. HLM is a “value added” methodology. That is, after controlling for 
students’ initial achievement and characteristics and accounting for variance at the student and school 
levels, researchers can assess the “value added” by the treatment. The analyses to follow contrast the 
achievement of immersion and control schools for three student cohorts: 

• Cohort 1, before and after three years of project implementation (sixth to eighth grade), 

• Cohort 2, before and after two implementation years (sixth to seventh grade), and 

• Cohort 3, before and after one implementation year (sixth grade). 

Immersion effects for Cohort 1 are estimated for TAKS reading, mathematics, social studies, and 
science T scores. For Cohort 2, effects are estimated for TAKS reading, mathematics, and writing 
T scores. Researchers used three-level HLM growth models to examine changes in students’ TAKS 
reading and mathematics achievement over time. For TAKS social studies, science, and writing, 
students had scores for only two time points, so data analysis involved two-level HLM models. 
Similarly, the effects of immersion on Cohort 3 students’ TAKS reading and mathematics T scores 
were analyzed using two-level HLM models. (See Appendix E for technical detail on the HLM 
models.) 

The availability of achievement data for three student cohorts allowed researchers to evaluate program 
effects by examining the importance of group differences, and the replicability or truth of group 
differences across cohorts and outcome measures (e.g., Cohen, 1994; Schmidt, 1996). Since small 
effects are noteworthy when evidence indicates that effects are replicable, we have reported effects as 
statistically significant at less conservative levels {p < .10) when findings provide evidence of 
important trends. 

TAKS Reading 

Cohorts 1 and 2 

TAKS reading achievement growth trajectories were estimated for Cohorts 1 and 2 students in 
immersion and control schools. Three-level HLM growth models examined the extent to which 
student achievement varied across time, students, and schools. Given the complexity of interpreting 
growth models, we constrained our final models to include school and student predictors that exhibited 
strong associations with achievement (i.e., school and student poverty). In the HLM growth model, 
level 1 is a repeated-measures model (i.e., TAKS assessment time within students) that captures the 
key features of growth (e.g., initial status, rate of change). Time is the point at which students 
completed assessments each spring (Cohort 1,0 = 2004, 1 = 2005, 2 = 2006, 3 = 2007; Cohort 2, 

0 = 2005, 1 = 2006, 2 = 2007). 

The between-students model (level 2) modeled differences between students in features of growth 
(e.g., initial status, rate of change), after adjusting for students’ economic status (1 if economically 
disadvantaged [i.e., eligible for the federal free- and reduced-price lunch program], 0 if not). At the 
school level (level 3), we examined how students’ initial status and growth varied across schools as a 
function of school-level random effects, as well as school conditions, including group membership 


67 



(1 for immersion, 0 for the control group) and school poverty (percentage of economically 
disadvantaged students attending a school). School poverty rates ranged from 31% to 100%, with a 
mean of 69.8%. Thus, we hypothesized that being in an immersion school is positively related to 
students’ growth in achievement, after controlling for the poverty level of the school. 

Separate HLM growth models were used to determine the effects of immersion on Cohort 1 and 
Cohort 2 students’ growth in TAKS reading achievement (Table 6.5). Growth models estimated 
school mean rates of change for immersion and control students, as well as the separate effects of 
student economic disadvantage and the school poverty concentration on reading. Analyses for 
Cohort 1 involved 1,380 immersion and 1,613 control students. Comparison groups had nearly 
equivalent proportions of students included in longitudinal analyses (58.9% for immersion and 57.9% 
for control). Cohort 2 analyses involved 1,546 immersion and 1,725 control students. As with 
Cohort 1, analyses involved nearly equal proportions of students across groups (58.3% for immersion 
and 60.7% for control). 


Table 6.5. HLM Statistics for Cohort 1 and Cohort 2 Students: 
Effects of Immersion on TAKS Reading Achievement Growth Rates 



Cohort 1 (Eight Graders) 
N = 2,993 

Cohort 2 (Seventh Graders) 
N = 3,271 

Dependent variable 
and predictor 

Gamma 

Coefficient 

/-value 

Gamma 

Coefficient 

/-value 

Initial mean status 
(2004/2005 TAKS T score) 

54.002 

76.89*** 

52.770 

100.87*** 

Immersion 3 

-1.346 

-1.8% 

-0.488 

-0.81 

School poverty 

-6.504 

_449*** 

-8.049 

_5 77 *** 

Economic disadvantage 

-6.170 

_9 40*** 

-5.591 

-9 33*** 

Growth rate 

-0.369 

- 2 . 86 ** 

-0.155 

-0.85 

Immersion 3 

0.212 

1.45 

0.388 

1.66 

School poverty 

0.860 

1 .75* 

0.905 

1.30 

Economic disadvantage 

0.536 

4 Q7*** 

0.283 

1.56 


f p < .10. *p < .05. **p < .01. ***p < .001. 

3 Immersion students had significantly lower initial TAKS reading scores. A latent variable 
regression, controlling for the effect of this initial difference on the growth rate, indicated that the 
difference between the original and adjusted immersion coefficients was not significant (the 
difference divided by the standard error of the difference = 1.30). 

As Table 6.5 shows, the initial mean TAKS reading status for the Cohort 1 reference group (an 
economically advantaged eighth grader in a control school with an average level of school poverty) is 
estimated at 54.00 (the mean 2004 TAKS reading T score). The coefficient representing immersion 
(-1.346) shows that students in immersion schools had lower initial TAKS reading T scores (52.66) 
than control students. Considering that differences among schools in students’ initial achievement may 
be related to subsequent rates of change, we used statistical tests to establish that those differences did 
not affect the estimations of student growth. Coefficients for initial status also showed that 
economically disadvantaged students and students attending schools with above average levels of 
poverty stalled behind their more advantaged counterparts in reading ability (-6.17 and -6.50 T-score 
points, respectively). 

After controlling for student and school levels of poverty, results show there was no statistically 
significant effect of immersion on Cohort 1 students’ growth rate for TAKS reading. Reading 
achievement for advantaged students in control schools (with average poverty) decreased by -0.37 
T-score point per year (significantly negative coefficient, -0.369). The positive coefficient for 
immersion (0.212) indicates that reading scores for advantaged students in immersion schools (with 
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average poverty) decreased at a slower rate (-0.16 T-sccre point per year) compared to control-group 
students (-0.369 + 0.212 = -0.157). Economically disadvantaged eighth graders at both immersion and 
control schools grew in reading achievement at significantly faster rates than their more advantaged 
peers (0.38 T-score point per year for immersion and 0.17 T-score point for control students). 

TAKS reading outcomes for Cohort 2, similarly, showed no statistically significant effect of 
immersion on seventh graders’ reading achievement. The reading T scores of advantaged seventh 
graders in control schools with average poverty decreased (-0.16 T-score point per year), while the 
scores for advantaged students in immersion schools increased (0.23 T-score point per year). 
Economically disadvantaged seventh graders at both immersion and contrcd schools grew in reading at 
a slightly faster rate than their more advantaged students. 

Figures 6.1 and 6.2 illustrate the estimated mean TAKS reading growth trajectories for advantaged and 
disadvantaged Cohort 1 and Cohort 2 students, respectively, by school comparison group. 
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Figure 6.1. Estimated mean TAKS reading achievement growth trajectories for Cohort 1 
economically advantaged and disadvantaged student groups in immersion and control schools. 
Differences between immersion and control groups are statistically insignificant. 
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Figure 6.2. Estimated mean TAKS reading achievement growth trajectories for Cohort 2 
economically advantaged and disadvantaged student groups in immersion and control schools. 
Differences between immersion and control groups are statistically insignificant. 


Cohort 3 

We analyzed the effects of immersion on Cohort 3 students’ TAKS reading T scores using a two-level 
HLM model (see Table 6.6). In the student-level model (level 1), 2007 TAKS reading T scores were 
regressed on 2006 reading scores, gender (1 if female, 0 if male), minority status (1 if Hispanic, 0 if 
not; 1 if African American, 0 if not), and economic status (1 if economically disadvantaged, 0 if not). 
A school-level model (level 2) estimated whether immersion schools had higher achievement than 
control schools, after controlling for initial achievement, student demographic characteristics, and 
school poverty. The immersion variable identified the comparison groups (I = immersion, 

0 = control). School poverty was a continuous variable depicting the concentration of economically 
disadvantaged students in a school. Analyses involved 1,778 immersion students and 1,991 control 
students, with similar proportions of students included in analyses (68.5% and 70.1%, respectively). 


Table 6.6. HLM Statistics for Cohort 3 (Sixth Graders): 
Effect of Immersion (Fixed) on TAKS Reading Achievement 



TAKS Reading 


N = 3,769 

Dependent variable 

Gamma 


and predictor 

Coefficient 

r-value 

Intercept (2007 TAKS T score) 

49.140 

145.71*** 

Immersion 

0.082 

0.33 

School poverty 

1.591 

1.82 

Female 

1.385 

7 41*** 

African American 

-1.486 

-4 23*** 

Hispanic 

-1.075 

-3.88** 

Economic disadvantage 

-0.539 

-2.81** 

2006 TAKS T score 

0.523 

34 72*** 


*p < .05. **p < .01. ***p < .001. 
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TAKS reading outcomes for sixth graders reported in Table 6.6 show that after controlling for 
students’ prior reading achievement, demographic characteristics, and the level of school poverty, 
there were no statistically significant differences in the 2007 TAKS reading T scores for students in 
immersion and control schools. Similar to Cohorts 1 and 2, the immersion coefficient was positive 
(about 0.08 T- score points). Students’ individual characteristics, however, were the strongest 
predictors of reading achievement. Female students had significantly higher TAKS reading T scores 
than males, whereas African American, Flispanic, and economically disadvantaged students had 
significantly lower TAKS reading scores than other students. 

TAKS Mathematics 

Cohorts 1 and 2 

Similar to reading, we estimated the TAKS mathematics achievement growth trajectories for 
Cohorts 1 and 2 students in immersion and control schools (Table 6.7). Three-level FILM growth 
models were used to examine the extent to which mathematics achievement varied across time (the 
point at which students completed TAKS assessments each spring), students, and schools. Results for 
Cohort 1 students show that control students initially had an estimated mean mathematics T score of 
53.02, whereas immersion student began with a lower estimated mathematics score (51.82). 
Economically disadvantaged students and students attending schools with above average levels of 
poverty stalled significantly behind their more advantaged peers in math ability (-4.487 and -4,724 
T-score points, respectively). 


Table 6.7. HLM Statistics for Cohort 1 and Cohort 2 Students: 

Effects of Immersion on TAKS Mathematics Achievement Growth Rates 



Cohort 1 (Eight Graders) 
N = 3,013 

Cohort 2 (Seventh Graders) 
N = 3,310 

Dependent variable 
and predictor 

Gamma 

Coefficient 

t- value 

Gamma 

Coefficient 

r- value 

Initial mean status 
(2004/2005 TAKS T score) 

53.019 

70.83*** 

52.306 

95.59*** 

Immersion 3 

-1.201 

-1.36 

-1.029 

-1.54 

School poverty 

-4.724 

-2.50* 

-4.373 

-2.39* 

Economic disadvantage 

-4.487 

-8.60*** 

-4.492 

-7 80*** 

Growth rate 

-0.181 

-1.15 

-0.444 

-1.65 

Immersion 3 

0.582 

1.95^ 

0.708 

1.78* 

School poverty 

1.488 

1.67 

0.317 

0.31 

Economic disadvantage 

0.025 

0.29 

0.044 

0.24 

Economic x Immersion 

-0.408 

-1.90 t 

— 

- 


'p < .10. *p < .05. **p < .01. ***p < .001. 


After controlling for student and school levels of poverty, the effect of immersion on students’ growth 
rate for TAKS mathematics scores was statistically significant at the p < .10 level. Estimated 
mathematics achievement for economically advantaged students in immersion schools (with average 
poverty) increased by about 0.40 T-score point per year (coefficient of 0.582), while the math scores of 
their control-group counterparts decreased by about 0.18 T-score point per year (coefficient of -0.181). 
A significant interaction effect between students’ socioeconomic status and immersion was also 
detected. Economically disadvantaged students in immersion schools grew in mathematics 
achievement at a significantly slower rate (about 0.02 T-score point per year) than more advantaged 
immersion students (about 0.40 point per year). In contrast, economically advantaged and 
disadvantaged control students had comparably negative growth trends (-0.18 and -0.16 T-score point, 
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respectively). Figure 6.3 illustrates the estimated mean TAKS mathematics growth trajectories for 
Cohort 1 advantaged and disadvantaged students at immersion and control schools. 



— ♦ — Immersion: Advantaged Students — ■ — Immersion: Disadvantaged Students 

— -0 — Control: Advantaged Students — -B — Control: Disadvantaged Students 


Figure 6.3. Significant immersion effect and cross-level interaction between immersion and 
economically disadvantaged status (p < .10) for Cohort 1 TAKS mathematics achievement (in 
schools with average levels of poverty). 

TAKS mathematics outcomes for Cohort 2 revealed a statistically significant effect of immersion on 
seventh graders’ math achievement for both advantaged and disadvantaged students at the p < .10 
level. The mathematics T scores of advantaged seventh graders in immersion schools (with average 
poverty) increased (0.26 T-score point per year), while the scores for advantaged students in control 
schools decreased (-0.44 T-score point per year). Similarly, the math scores for economically 
disadvantaged seventh graders at immersion schools increased (0.30 T-score point per year), whereas 
disadvantaged control-group students had a negative growth trend (- 0.40 T-score point per year). 
Figure 6.4 illustrates the estimated mean TAKS mathematics growth trajectories for Cohort 2 
advantaged and disadvantaged students at immersion and control schools. 
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Figure 6.4. Significant immersion effect (p < .10) for Cohort 2 students’ TAKS mathematics 
achievement (in schools with average levels of poverty). 

Cohort 3 

We analyzed the effects of immersion on Cohort 3 sixth graders’ mathematics scores using a two-level 
HLM model (Table 6.8). In the student-level model, 2007 TAKS mathematics T scores were regressed 
on 2006 mathematics scores, gender, minority status, and economic status. A school-level model 
estimated whether immersion schools had higher TAKS achievement scores than control schools, after 
controlling for initial achievement, student demographic characteristics, and school poverty 
(percentage of economically disadvantaged students). Analyses involved 1,782 immersion students 
and 2,031 control students, with similar proportions of students included in analyses across groups. 


Table 6.8. HLM Statistics for Cohort 3 (Sixth Graders): 

Effect of Immersion (Fixed) on TAKS Mathematics Achievement 


Dependent variable 
and predictor 

TAKS Mathematics 
N = 3,813 

Gamma 

Coefficient 

f-value 

Intercept (TAKS T score) 

48.799 

99.43*** 

Immersion 

0.282 

0.44 

School poverty 

0.265 

0.16 

Female 

0.944 

4.05*** 

African American 

-1.309 

-3.69** 

Hispanic 

-0.486 

-1.99* 

Economic disadvantage 

-1.064 

-4 23*** 

2006 TAKS T score 

0.608 

28.98*** 

Pretest x Immersion 

0.082 

3.41** 


*p < .05. **p < .01. ***p < .001. 
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TAKS mathematics outcomes for sixth graders reported in Table 6.8 show that after controlling for 
students’ prior TAKS achievement, demographic characteristics, and the level of school poverty, there 
was no significant main effect of immersion on sixth graders’ 2007 TAKS mathematics T scores. The 
immersion effect was positive but not by a significant margin (about 0.28 T-score point). However, a 
significant interaction effect was detected, which acted through the TAKS pretest score. Thus, the 
magnitude of the immersion effect depended upon students’ prior math achievement. Other factors 
being equal, having higher TAKS 2006 pretest scores predicted larger gaps in the posttest mathematics 
scores (2007) favoring immersion students. Thus, for TAKS mathematics, immersion had a stronger 
and statistically significant effect on higher achieving students {p < .01). 

Figure 6.3 illustrates the interaction effect for average immersion and control students. As an example, 
students in immersion schools with pretest TAKS mathematics T scores near 30.0 would have posttest 
TAKS scores about 1.30 T-score points below control students. On the other hand, immersion students 
who had pretest mathematics T scores of about 60.0 would have posttest scores nearly 2.00 T-score 
points higher than control students. 



Figure 6.5. Significant immersion effect (p <=.01 ) on Cohort 3 TAKS mathematics achievement, 
which acts through the pretest score. 

TAKS Social Studies and Science 

Cohort 1 

Cohort 1 students also completed TAKS science and social studies assessments in spring 2007. For 
science, eighth graders completed a baseline measure as fifth graders in 2004. The TAKS social 
studies assessment is administered for the first time in eighth grade. Since there was no baseline 
measure, we used students’ 2006 TAKS reading score as a control for academic achievement. The 
effects of immersion on Cohort 1 students’ science and social studies scores were analyzed using 
two-level HLM models (see Table 6.9). In the student-level model (level 1), students’ 2007 T scores 
were regressed on students’ baseline scores, gender, minority status, and economic status. A 
school-level model (level 2) was used to determine whether students in immersion schools had higher 
TAKS science and social studies scores than control-group students in spring 2007, after adjusting for 
initial achievement, student demographic characteristics, and school poverty. The immersion variable 
identified the comparison groups (a value of 1 for an immersion school and 0 for control). School 
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poverty was a continuous variable, with a mean of 69.8%, indicating the percentage of economically 
disadvantaged students in a school. Analyses for TAKS science involved 1,367 immersion students 
and 1,600 control students, with similar proportions of students included in analyses across groups 
(58.3% for immersion and 57.4% for control). Analyses for TAKS social studies involved 1,443 
immersion students and 1,700 control students, with similar proportions of students included in 
analyses across groups (61.6% for immersion and 61.0% for control). 


Table 6.9. HLM Statistics for Cohort 1 (Eighth Graders): 

Effect of Immersion (Fixed) on TAKS Science and Social Studies Achievement 



TAKS Science 
N = 2,967 

TAKS Social Studies 
N = 3,143 

Gamma 

Coefficient 

r-value 

Gamma 

Coefficient 

r-value 

Intercept (TAKS T score) 

52.269 

91.89*** 

49.525 

45.93*** 

Immersion 

-0.238 

-0.36 

0.574 

0.95 

School poverty 

0.167 

0.09 

4.607 

2.78** 

Female 

-1.203 

-5.90*** 

-2.537 

-10.32*** 

African American 

-2.052 

-2.57* 

-1.851 

-5.31*** 

Hispanic 

-1.712 

-4.47*** 

-2.033 

-3 99 *** 

Economic disadvantage 

-1.518 

_4 41 *** 

-1.614 

-4.35*** 

2004/2006 TAKS T score a 

0.681 

23.72*** 

0.539 

35.30*** 

Pretest x Immersion 

-0.084 

-1.90 1 

- 



f p < .10. *p < .05. **p < .01. ***p < .001. 

a The pre-measure for science is the 2004 TAKS science score; the pre-measure for TAKS social studies 
is the 2006 TAKS reading score. 


Science outcomes for Cohort 1 eighth graders show that after controlling for students’ prior TAKS 
achievement, demographic characteristics, and the level of school poverty, there was no significant 
main effect of immersion on eighth graders’ 2007 TAKS science T scores. In fact, the immersion 
effect was negative (-0.24 T- score point). Additionally, a significant interaction effect was detected, 
which acted through the TAKS 2004 science pretest score. The magnitude of the immersion effect 
depended upon students’ prior achievement. Other factors being equal, having higher TAKS science 
pretest scores (2004) predicted larger gaps in the 2007 science scores favoring control students. Thus, 
for higher achieving students in immersion schools, there was a significantly negative effect on 
science achievement (a -0.084 T-score point decrease for each T-point increase in prior science 
achievement). 

Results for TAKS social studies, showed that after controlling for students’ 2006 TAKS reading 
achievement, demographic characteristics, and the level of school poverty, there was no statistically 
significant difference between the 2007 TAKS social studies T scores for immersion and control 
students. In contrast to science, the immersion effect was positive (0.57 T-score point) but not by a 
statistically significant margin. 

Across both immersion and control schools, economically disadvantaged students had significantly 
lower TAKS science scores (-1.52 T-score points) and social studies scores (-1.61 T-score points) than 
their more affluent counterparts. And, minority students (African American and Hispanic) had 
significantly lower scores than other students. Unexpectedly, female students had significantly lower 
science and social studies scores than males (1.20 T-score point and 2.54 T-score point, respectively). 
This contrasts with prevailing trends showing that females outperform males on TAKS reading and 
mathematics assessments. 
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TAKS Writing 
Cohort 2 

Cohort 2 students completed the TAKS writing assessment as fourth graders in 2004 and again as 
seventh graders in 2007. We used a two-level HLM model to estimate the effects of immersion on 
students’ writing scores (see Table 6.10). In the student-level model (level 1), students’ 2007 writing 
T scores were regressed on 2004 writing scores (data from two years prior to students’ involvement in 
the immersion project), gender, minority status, and economic status. A school-level model (level 2) 
predicted whether students in immersion schools had higher 2007 TAKS writing T scores than control- 
group students, after adjusting for initial achievement, student demographic characteristics, and school 
poverty. HLM analyses involved 1,457 immersion students and 1.631 control students. 

Table 6.10. HLM Statistics for Cohort 2 (Seventh Graders): 

Effects of Immersion (Fixed) on TAKS Writing Achievement 


Dependent variable 
and predictor 

TAKS Writing 
N = 3,088 

Gamma 

Coefficient 

f-value 

Intercept (TAKS T score) 

51.181 

100.44*** 

Immersion 

-0.283 

- 0.66 

School poverty 

0.315 

0.24 

Female 

1.409 

5 94 *** 

African American 

-1.141 

-2.95• ** 

Hispanic 

-1.355 

-4.47*** 

Economic disadvantage 

- 1.210 

-3.36** 

Spring 2004 T score 

0.453 

28.59*** 


**p < . 01 . ***p< . 001 . 


Results for Cohort 2 students show that after controlling for students’ pretest writing scores, student 
demographic characteristics (gender, ethnicity, economic status), and campus poverty level, there was 
no statistically significant difference in the 2007 TAKS writing T scores for students in immersion and 
control schools. The immersion effect on writing was negative (about -0.28 T-score point lower than 
for control-group students). Across both immersion and control schools, the demographic 
characteristics of students were strongly associated with TAKS writing achievement. Female students 
had significantly higher writing scores than males (about 1.41 T-score points), African American and 
Hispanic students had significantly lower writing scores than other students (-1.14 and -1.36 points, 
respectively), and economically disadvantaged students had significantly lower scores than their more 
affluent peers (-1.21 T-score points). 


Conclusions 

In the third project year, we examined the effects of immersion on Cohort 1 students (eighth graders 
who attended schools for three years), Cohort 2 students (seventh graders who attended schools for 
two years), and Cohort 3 students (sixth graders who attended one school year). Key findings are the 
following. 

• TAKS reading. After controlling for student and school poverty, there were no statistically 
significant effects of immersion on the TAKS reading growth rates for either Cohort 1 or 

Cohort 2 students. The immersion effects were positive but not by significant margins. Across 

both student cohorts, positive mean growth trajectories showed that economically 
disadvantaged students grew in reading achievement at faster rates than their more affluent 
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peers. This growth provided a substantial boost in reading achievement for economically 
disadvantaged immersion students in Cohorts 1 and 2. 

For Cohort 3, after controls for students’ prior achievement, demographic characteristics, and 
school poverty, there was no statistically significant effect of immersion on students’ 2007 
TAKS reading T scores. Similar to the other cohorts, the immersion coefficient for reading 
was positive. 

• TAKS mathematics. After controlling for student and school poverty, technology immersion 
had a statistically significant effect on TAKS mathematics growth rates for both Cohorts 1 and 
2 students. The TAKS mathematics scores of immersion students increased across years, 
whereas scores for control students decreased. For Cohort 1 , a significant interaction effect 
revealed that economically advantaged students in immersion schools grew at a significantly 
faster rate than disadvantaged students. For Cohort 2, economically advantaged and 
disadvantaged immersion students had similarly positive TAKS mathematics growth trends 
that significantly outpaced their control-group counterparts. 

For Cohort 3, after controlling for students’ prior achievement, demographic characteristics, 
and school poverty, there was a statistically significant effect of immersion that acted through 
students’ pretest scores. As TAKS pretest scores increased, the mathematics achievement gap 
for 2007 TAKS scores favoring immersion students over control widened. Thus, immersion 
had a significantly positive effect on mathematics scores for higher achieving sixth graders. 

• TAKS social studies. After controlling for Cohort 1, eighth graders’ reading achievement 
(seventh grade), demographic characteristics, and school poverty, there was no statistically 
significant effect of immersion on 2007 TAKS social studies scores. The immersion effect 
was positive but not by a statistical margin. 

• TAKS science. After controlling for prior science achievement, demographic characteristics, 
and school poverty, there was no statistically significant effect of immersion on Cohort 1, 
eighth graders’ 2007 TAKS science achievement. The immersion effect was negative. 
Additionally, there was a statistically significant interaction between pretest and posttest 
scores. As students’ TAKS science pretest scores increased, the achievement gap favoring 
control students over immersion widened for 2007 TAKS science scores. Thus, immersion 
had a negative effect on the science scores of higher achieving eighth graders. 

. TAKS writing. After controlling for Cohort 2 seventh graders’ pretest writing scores (fourth 
grade), demographic characteristics, and campus poverty, there was no statistically significant 
difference in the TAKS writing scores for immersion and control students. The immersion 
effect was negative but not by a significant margin. 
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7. Factors Associated with Implementation and Outcomes 


In Chapter 3, we reported findings on the implementation of technology immersion for the second and 
third project years (2005-06 and 2006-07, respectively). Findings revealed some third-year 
improvements relative to school supports for implementation as well as the extent of teachers’ 
Classroom Immersion. Student Access and Use, in contrast, remained stable across years and 
generally fell short of expectations. Implementation evidence, on the other hand, showed wide 
variation across schools and classrooms. For the third year, we estimated that about a quarter of 
middle schools (5) with complete data sets had a much stronger presence of the immersion 
components that more nearly approximated expected standards. Given the variation in implementation 
from school-to-school and from classroom-to-classroom, we report in this chapter on investigations of 
factors that arc associated with the implementation levels of technology immersion, and the 
relationships between implementation levels and student academic achievement. Implementation is 
measured as the fidelity with which technology immersion components and related elements attained 
the model’s envisioned ideal (see implementation indicators in Exhibit 7.1). 


Exhibit 7.1. Implementation Indicators for Technology Immersion 

Immersion Support Index is an aggregate score for school-level indicators of support for technology 
immersion. 

■ Leadership is a measure of administrative leadership for technology. 

■ Teacher Support is a measure of teachers’ commitment to immersion. 

■ Parent and Community Support is a measure of support for the school’s technology efforts. 

■ Technical Support is a measure of the extent to which technical support alleviates problems that 

create barriers to immersion. 

■ Professional Development is an aggregate indicator of the quality of campus professional 

development as measured by four elements: Contact Hours , Classroom Support , Content Focus, and 
Coherence. 

Classroom Immersion Index is an aggregate score for teacher-level immersion indicators. 

■ Technology Integration is a measure of a teacher’s ideological orientation towards classroom 

technology immersion. 

■ Learner-Centered Instruction is a measure of a teacher’s ideological orientation towards student- 

centered learning practices. 

■ Student Activities is a measure of the frequency of students’ use of technology resources in a 

teacher’s classroom. 

■ Communication is a measure of a teacher’s technology-based communications with students, parents, 

and peers. 

■ Professional Productivity is a measure of a teacher’s use of technology for professional activities. 

Student Access and Use Index is an aggregate score for student-level immersion indicators. 

■ Laptop Access Days is a measure of the extent to which a student has access to a laptop throughout 

the school year. 

■ Core-Content Learning is a measure of the frequency that a student reports using technology for 

learning in core-subject classes. 

■ Home Learning is a measure of the extent that a student uses a laptop for core-subject homework 

(language arts [reading/writing], social studies, science, and math) or to play games to learn outside 
of school. 

Implementation Index is an implementation score for each school, which is an aggregate score for the three 
implementation components described above. 

Note. Implementation indices are z scores with a mean of 0 and standard deviation of 1 .0. 


79 





For analyses of the factors associated with implementation and outcomes, we used standardized 
implementation indicators (z scores with a mean of 0 and a standard deviation of 1 .0) that could be 
analyzed individually or aggregated to generate component scores and an overall implementation 
score. Analyses involved indicators that assessed school supports for immersion (Immersion Support 
Index), the extent of teachers’ classroom immersion (Classroom Immersion Index), and the extent of 
students’ technology access and use (Student Access and Use Index). 

Factors Associated with Model Implementation 

In the section below, we explore relationships among various implementation components and 
examine whether particular support mechanisms or school characteristics are associated with schools’ 
levels of Classroom Immersion and Student Access and Use. The strength of relationships between 
implementation levels (mean campus z scores) and school characteristics (campus enrollment, 
percentages of minority and economically disadvantaged students, and percentages of students passing 
TAKS tests) are examined individually through bivariate correlations. 

Immersion Components 

Table 7. 1 displays the correlations between seven components of technology immersion, with 
statistically significant coefficients denoted in bold. As anticipated, teachers’ perceptions of their 
school’s administrative leadership was significantly associated with their perceived supports for 
implementation, including their collective support for technology innovation, views on parent and 
community support, prevalence of technical support, and robustness of professional development. 
Reasonably, teachers’ support for technology innovation was significantly related to other support 
mechanisms. The strength of administrative leadership and the intensity of campus professional 
development supporting immersion were significantly associated with higher levels of Classroom 
Immersion. 

Table 7.1. Correlations of Technology Immersion Components 

Components of Technology Immersion 


Component 

Immersion Support 

Classroom 

Immersion 

Student 

Access/Use 

Leader- 

ship 

Teacher 

Support 

Parent 

Support 

Technical 

Support 

PD 

Leadership 

1.00 





Teacher Support 

.81** 

1.00 




Parent/Community Support 

.58** 

.68** 

1.00 





Technical Support 

.49* 

.59** 

.56** 

1.00 




Professional Development 

.64** 

.59** 

.50* 

.20 

1.00 



Classroom Immersion 

.57** 

.41 

.25 

.14 

' 77 ** 

1.00 


Student Access and Use 

.28 

.48* 

.45 

.54* 

.18 

.18 

1.00 


*p < . 05 . **p < . 01 . 

Note. IV = 21 immersion schools. PD = Professional Development. 


Teachers’ overall support for technology innovation (Teacher Support) was positively associated with 
the school’s level of Student Access and Use. Additionally, students’ reported access to and use of 
laptops was significantly correlated with the quality of technical support for immersion. In general, 
students had more robust technology experiences when all teachers at a school more strongly 
supported technology innovation and technical supports addressed maintenance issues that created 
barriers to student laptop use. School leaders played a key role in providing supports for teachers’ 
classroom immersion efforts. 
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School Characteristics 


We also explored the relationship between implementation components and school characteristics (see 
Table 7.2). Middle-school campus characteristics included the average student enrollment, percentage 
of minority students (African American and Hispanic), school poverty (percentage of economically 
disadvantaged students as measured by eligibility for federal free- and reduced-price lunches), and 
achievement (percentage of grades 6 to 8 students passing all TAKS tests in spring 2007). Results 
showed that school size was negatively associated with implementation levels for most technology 
immersion components. That is, schools with larger student enrollments tended to have somewhat 
lower levels of implementation than schools with fewer students, although the negative relationship 
was significant for only one component. Teachers at larger schools reported significantly lower levels 
of technical support, indicating that technical problems posed a greater barrier to technology 
immersion at such schools. 


Table 7.2. Correlations of Technology Immersion Components and School Characteristics 


Immersion Component 


Characteristics of Middle Schools 


Student 

Enrollment 

Percent 

Minority 

Students 

Percent 

Economically 

Disadvantaged 

Students 

Percent of 
Students 
Passing All 
TAKS Tests 

Leadership 

-.20 

-.14 

-.38 

.31 

Teacher Support 

-.24 

-.23 

-.45* 

.40 

Parent & Community Support 

-.37 

-.12 

-.33 

.50* 

Technical Support 

-.49* 

-.04 

-.13 

.27 

Professional Development 

.22 

.11 

-.09 

.00 

Classroom Immersion 

.18 

-.07 

-.16 

-.12 

Student Access & Use 

-.31 

-.18 

-.34 

.34 

Implementation Index 

-.16 

-.17 

-.41 

.23 


*p < .05. **p < .01. 

Note. N = 21 immersion schools. 


Higher percentages of minority students (African American and Hispanic) showed a weakly negative 
relationship with implementation components. Higher percentages of economically disadvantaged 
students at a school had an even stronger negative relationship with implementation levels, although 
only one indicator reached statistical significance. Schools with more disadvantaged populations had 
significantly lower levels of teacher support for the implementation of innovative technology 
practices. In contrast to the negative relationships between school demographic characteristics and 
implementation, the school’s achievement context was positively associated with nearly all of the 
implementation indicators, although correlations were generally low. Teachers’ reported level of 
Classroom Immersion was the only immersion component that was negatively correlated with 
achievement. On the other hand, teachers’ perceptions of parent and community support for 
technology was significantly and positively associated with students’ TAKS achievement. 

Classroom Immersion 

To further understand teachers’ perspectives, an additional analysis examined the relationships among 
support components, school characteristics, and elements of core-subject teachers’ Classroom 
Immersion. Correlation coefficients presented in Table 7.3 showed generally low associations among 
variables, with some positive and some negative relationships. Still, a few statistically significant 
findings surfaced. Classroom Immersion elements gauging the strength of teachers’ ideological 
agreement with technology innovation and constructivist practices (Technology Integration and 
Learner-centered Instruction) were significantly related to teachers’ perceptions of the viability of 
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various support components, including administrative leadership, parent and community support, and 
professional development. Teachers’ perceptions of the robustness of professional development 
showed the strongest relationship with the elements of Classroom Immersion and the composite index. 


Table 7.3. Correlations of Support Components and School Characteristics by Elements of 
Classroom Immersion 


Indicator/Characteristic 

Core-Subject Teachers’ Classroom Immersion j 

Technology 

Integration 

Learner- 

Centered 

Instruction 

Student 

Activities 

Communi- 

cation 

Professional 

Productivity 

Classroom 

Immersion 

Index 

Leadership 

.65** 

.54* 

.65** 

.24 

.15 

.57** 

Teacher Support 

.59** 

.43* 

.51* 

.10 

-.04 

.41 

Parent & Community Support 

.44* 

.36 

.32 

.00 

-.14 

.25 

Technical Support 

.28 

.21 

.27 

-.07 

-.13 

.14 

Professional Development 

.80** 

.75** 

.61** 

.49* 

.35 


School enrollment 

.07 

.02 

.12 

.29 

.30 

.18 

% minority students 

-.13 

.05 

-.08 

-.10 

.07 

-.07 

% Disadvantaged students 

-.16 

.08 

-.14 

-.28 

.03 

-.16 

% Students pass all TAKS 

.02 

-.13 

-.16 

-.10 

-.23 

-.12 


*p < .05. **p < .01. 

Note. N = 21 immersion schools. 


Also, as illustrated in Table 7.3, significant associations linked the strength of administrative 
leadership with greater teacher ideological affiliations, frequency of Student Activities, and the overall 
level of Classroom Immersion. On the other hand, there were generally weak relationships between 
the characteristics of schools and the elements of Classroom Immersion. Moreover, the school’s mean 
achievement on TAKS was negatively associated with teachers’ implementation levels for almost all 
of the Classroom Immersion elements. 

Student Access and Use 

Correlations for students’ reported levels of technology access and use showed important trends 
(Table 7.4). First, the strength of the composite Student Access and Use Index was significantly 
related to stronger levels of teacher and technical support for implementation. And, although 
correlations were statistically insignificant, the index was negatively associated with larger school size 
and higher percentages of minority and economically disadvantaged students. 


Table 7.4. Correlations for Support Components and School Characteristics by Elements 
of Student Access and Use 


Indicator/Characteristic 

Student Technology Access and Use 

Laptop 
Access Days 

Classroom 

Learning 

Home 

Learning 

Student 

Access/Use 

Index 

Leadership 

-.33 

.36 

.17 

.28 

Teacher Support 

-.10 

.42 

.37 

.48* 

Parent & Community Support 

.18 

.40 

.26 

.45 

Technical Support 

.12 

.50* 

.34 

.54* 

Professional Development 

-.27 

.07 

.31 

.18 

School Enrollment 

-.31 

-.51* 

.14 

-.31 

% Minority Students 

-.08 

-.38 

.16 

-.18 

% Disadvantaged students 

-.06 

-.39 

-.11 

-.34 

% Students pass all TAKS 

.19 

.27 

.15 

.29 


*p < .05. **p <.01. 

Note. Data are for 19 immersion schools. Two schools did not submit student surveys. 
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Second, unexpectedly, the perceived strength of campus technical support was not strongly related to 
the number of days that students had their laptops available for use; however, technical support was 
significantly associated with the frequency with which students used their laptops for learning in core- 
subject classrooms. There were no significant associations between teachers’ perceived supports for 
immersion and students’ tendencies to use their laptops for learning at home (i.e., homework and 
learning games). However, the robustness of a student’s technology access and use was negatively 
associated with the characteristics of the school that he or she attended. Students attending larger 
schools and schools with larger minority and economically disadvantaged populations reported 
generally lower levels of technology access and use. On the other hand, students’ reported technology 
access and use was positively, though weakly, associated with campus academic achievement. 

Factors Associated with Student Outcomes 

An additional investigation of the associations between implementation fidelity and student academic 
outcomes involved data for individual students and their teachers. We used a series of two-level 
hierarchical linear models (HLM), in which students were nested within teachers’ classrooms, to 
investigate whether the levels of implementation for two teacher-related implementation components 
(Immersion Support Index, Classroom Immersion Index) and one student-specific component (Student 
Access and Use Index) were significant predictors of students’ TAKS reading and mathematics scores. 
We analyzed the effects of implementation on academic achievement for Cohorts 1, 2, and 3 students. 

In the student-level model (level 1), 2007 TAKS T scores were regressed on 2006 TAKS T scores, the 
Student Access and Use Index (z score), economic status (0 if not disadvantaged, 1 if disadvantaged), 
African American status (0 if not African American, 1 if African American), Hispanic status (0 if not 
Hispanic, 1 if Hispanic) and gender (0 if male, 1 if female). The teacher-level model (level 2) 
investigated whether the Immersion Support Index (average campus z score) and Classroom 
Immersion Index (individual teacher z score) predicted higher 2007 TAKS scores, after adjusting for 
school poverty, students’ prior achievement and demographic characteristics, and Student Access and 
Use. We also investigated whether Student Access and Use predicted higher 2007 TAKS scores, after 
adjusting for initial achievement, student demographic characteristics, school poverty. Immersion 
Support, and Classroom Immersion. School poverty was a continuous variable indicating the 
percentage of economically disadvantaged students in a school, with a mean of 69.8%. Analyses for 
Cohorts 1, 2, and 3 involved approximately 1,200, 1,300, and 1,600 students, respectively, who were 
enrolled continuously in schools during three, two, and one project years. 

TAKS Reading 

Table 7.5 provides estimates of the effects of implementation on Cohorts 1, 2, and 3 students’ 2007 
TAKS reading T scores. We examined the effects of implementation for both students and teachers. 
First, at the teacher level, we investigated whether the strength of reading teachers’ campus support for 
implementation (Immersion Support) and their reported levels of Classroom Immersion were 
predictors of students’ reading achievement. Results for Immersion Support were mixed. After 
controlling for student variables (prior achievement, demographic characteristics, Student Access and 
Use) and other teacher variables (school poverty and Classroom Immersion), Immersion Support was 
a statistically significant and positive predictor of Cohort 1 eighth graders’ reading achievement. That 
is, controlling for other variables in the analysis, students whose teachers had average levels of 
Immersion Support (z = -0.1 1) had higher 2007 TAKS reading scores (1.34 T score points) than 
students with teachers having support that was approximately one standard deviation below average. 
Conversely, for Cohorts 2 and 3, there was no significant association between Immersion Support and 
students’ reading scores. Moreover, reading teachers’ level of Classroom Immersion, surprisingly, was 
an insignificant predictor of students’ TAKS achievement. In fact, for Cohorts 1 and 3, after adjusting 


83 



for other variables in the analysis, students who had reading teachers with average levels of Classroom 
Immersion had lower TAKS reading T scores (-0.23 and -0.71, respectively) than students with 
teachers having below average Classroom Immersion scores. 


Table 7.5. Hierarchical Regression Models Predicting the Effects of Implementation Components on 
TAKS Reading Achievement 



Cohort 1 
Eighth Graders 
N= 1,217 

Cohort 2 
Seventh Graders 
N= 1,297 

Cohort 3 
Sixth Graders 
N= 1,606 

Predictor 

Gamma 

Coefficient 

t-value 

Gamma 

Coefficient 

r-value 

Gamma 

Coefficient 

r-value 

Intercept 

49.803 

128.06*** 

50.984 

74.56*** 

48.869 

103.51*** 

Teacher-level predictors 

School poverty 

3.112 

1.76 

3.125 

0.97 

1.185 

0.75 

Immersion Support 

1.340 

5.62*** 

-0.129 

-0.25 

-0.030 

- 0.10 

Classroom Immersion 

-0.234 

-0.64 

0.688 

1.21 

-0.705 

- 2 . 02 + 

Student-level predictors 

Spring 2006 T score 

0.537 

23.03*** 

0.654 

19 77 *** 

0.532 

27 . 99 *** 

Student Access and Use 

0.542 

2.05* 

0.895 

2.56* 

0.523 

1 . 88 + 

Female 

0.394 

1.10 

-0.027 

-0.06 

0.785 

2.98** 

African American 

-0.285 

-0.31 

-2.562 

-2.91** 

-0.837 

- 1.21 

Hispanic 

-0.949 

-2.07* 

-2.443 

-2.84** 

-0.939 

-1.9F 

Eco. Disadvantaged 

-1.039 

-2.53* 

-0.625 

-1.13 

-0.266 

- 0.68 


< .10. *p < .05. **p < .01. ***p < .001. 

Note. Numbers of reading teachers: Cohort 1 = 39, Cohort 2 = 37, and Cohort 3=41. 


In contrast to results for teachers, the level of Student Access and Use (of technology) had a 
statistically significant positive relationship with reading achievement. Student-level results for the 
three cohorts show that after controlling for students’ prior reading achievement, demographic 
characteristics, and teacher-level variables (school poverty and implementation components), Student 
Access and Use was a significant predictor of students’ 2007 TAKS reading T scores. For Cohorts 1, 
2, and 3, the sizes of the technology access and use effect on TAKS reading achievement were 0.54, 
0.90, and 0.52 T-score points, respectively. As an example, after controlling for all of the other 
variables in the analysis, an economically advantaged, non-minority, male eighth grader with a score 
one standard deviation above average for Student Access and Use (z = 1.03), had a 0.54 T-score point 
higher TAKS reading score. Moreover, with each additional standard deviation increase in Student 
Access and Use, students’ reading achievement increased even more. 

Additionally, we conceptualized Student Access and Use as having multiple elements (Laptop Access 
Days, Core-Content Learning, and Fiome Learning), and thus, were interested in separately predicting 
variation for each element. Table 7.6 provides statistics for the FILM models used to predict each of 
the three elements. Findings revealed that Fiome Learning — which measured the extent of a student’s 
laptop use outside of school for homework in each of the four core-subject areas and for learning 
games — was the strongest implementation predictor of reading achievement. The Fiome Learning 
effect on TAKS reading scores was positive for Cohort 1 (0.31 T-score point) and statistically 
significant and positive for Cohort 2 (1.01 T-score point) and Cohort 3 (0.39 T-score point). In 
contrast, the number of days during the school year that students had laptops available for use (Laptop 
Access Days) was weakly associated with students’ reading achievement. The frequency that students 
reported using their laptops in their four core-subject classes (Core-Content Learning) was a non- 
significant positive predictor of achievement for Cohort 1 students, a significantly negative predictor 
for Cohort 2, and a negative predictor for Cohort 3. 
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Table 7.6. Hierarchical Regression Models Predicting the Effects of Implementation Components 
(including Elements of Student Access and Use) on TAKS Reading Achievement 


Teacher-Level 

Analysis 

Cohort 1 
Eighth Graders 

Cohort 2 
Seventh Graders 

Cohort 3 
Sixth Graders 

Gamma 

Coefficient 

r-value 

Gamma 

Coefficient 

r-value 

Gamma 

Coefficient 

r-value 

Intercept 

49.803 

130.90*** 

51.196 

75 77*** 

48.883 

104.32*** 

Teacher-level predictors 

School poverty 

3.287 

1.65 

2.916 

0.93 

1.026 

0.65 

Immersion Support 

1.325 

5.45*** 

0.143 

0.32 

-0.030 

-0.10 

Classroom Immersion 

-0.261 

-0.72 

0.762 

1.33 

-0.688 

-2.03* 

Student-level predictors 

Spring 2006 T score 

0.537 

22.03*** 

0.640 

18.44*** 

0.531 

27.80*** 

Laptop Access Days 

-0.033 

-0.32 

0.120 

0.41 

0.060 

0.22 

Core-Content Learning 

0.246 

0.81 

-0.485 

-2.00* 

-0.050 

-0.30 

Home Learning 

0.311 

1.37 

1.010 

^ 14*** 

0.394 

3.01** 

Female 

0.415 

1.19 

-0.142 

-0.32 

0.752 

2.82** 

African American 

-0.284 

-0.31 

-2.423 

-2.84** 

-0.785 

-1.16 

Hispanic 

-0.931 

-1.96* 

-2.630 

-3.05** 

-0.904 

-1 .86* 

Eco. Disadvantaged 

-1.054 

-2.60* 

-0.558 

-1.06 

-0.289 

-0.74 


*p < .05. **p < .01. ***p < .001. 

Note. Numbers of reading teachers: Cohort 1 = 39, Cohort 2 = 37, and Cohort 3=41. 


TAKS Mathematics 

We also estimated the effects of implementation on students’ 2007 TAKS mathematics T scores. Like 
reading, we examined implementation effects for students and teachers (Table 7.7). Teacher-level 
findings for Immersion Support varied across student cohorts. After controlling for other variables in 
the analysis. Immersion Support was a statistically significant and positive predictor of Cohort 1 
students’ mathematics achievement (comparable to effects for TAKS reading). That is, eighth graders 
who had mathematics teachers with average levels of Immersion Support (z. - -0.02) had higher 2007 
TAKS mathematics scores (1.46 T-score points, on average) than students who had teachers with 
Immersion Support that was about one standard deviation below average. For Cohorts 2 and 3, there 
was no significant association between Immersion Support and students’ mathematics achievement. 


Table 7.7. Hierarchical Regression Models Predicting the Effects of Implementation Components 
and TAKS Mathematics Achievement 



Cohort 1 
Eighth Graders 
N= 1,171 

Cohort 2 
Seventh Graders 
N= 1,382 

Cohort 3 
Sixth Graders 
N = 1,389 

Teacher-Level 

Analysis 

Gamma 

Coefficient 

r-value 

Gamma 

Coefficient 

r-value 

Gamma 

Coefficient 

r-value 

Intercept 

51.550 

83.22*** 

48.703 

85.70*** 

47.917 

80.95*** 

Teacher-level predictors 

School poverty 

8.713 

2.84** 

4.608 

1.55 

5.418 

1.55 

Immersion Support 

1.464 

2.99** 

0.643 

0.98 

-0.206 

-0.26 

Classroom Immersion 

-0.174 

-0.23 

0.621 

1 .76* 

0.292 

0.59 

Student-level predictors 

Spring 2006 T score 

0.707 

28.19*** 

0.731 

28.15*** 

0.689 

32.23*** 

Student Access and Use 

0.809 

3.34** 

0.864 

3.40** 

0.889 

2.97** 

Female 

-0.477 

-1.52 

-0.178 

-0.60 

1.025 

2.67** 

African American 

-1.037 

-1.63 

-0.553 

-1.11 

-1.219 

-1.34 

Hispanic 

-0.611 

-0.82 

-1.187 

-2.31* 

-0.667 

-0.98 

Eco. Disadvantaged 

-0.736 

-1.96* 

0.129 

0.31 

-0.518 

-0.94 


*p < .05. **p < .01. ***p < .001. 

Note. Numbers of mathematics teachers: Cohort 1 = 39, Cohort 2 = 40, and Cohort 3 = 33. 
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After statistical adjustments for the other variables in the analysis, mathematics teachers’ reported 
Classroom Immersion level, similar to reading teachers, had an insignificant relationship with 
students’ TAKS achievement. Teachers’ Classroom Immersion was a negative predictor of TAKS 
math achievement for Cohort 1 and a positive predictor for Cohorts 2 and 3. In contrast to teacher- 
related implementation indicators, students’ reported level of Student Access and Use was a 
statistically significant positive predictor of 2007 TAKS mathematics T scores for each of the student 
cohorts. Controlling for students’ prior math achievement, demographic characteristics, and teacher- 
level variables (implementation components as well as school poverty), the sizes of the Student 
Access and Use effects were 0.81, 0.86, and 0.89 T-score points for Cohorts 1, 2, and 3 students, 
respectively. For example, after controlling for other variables in the analysis, an economically 
advantaged, non-minority, male eighth grader with a score approximately one standard deviation 
above average (z - 1.02) for Student Access and Use had a 0.81 T-score point higher TAKS 
mathematics score than a student with an average Student Access and Use score (z, - 0.02). TAKS 
mathematics achievement increased incrementally as students’ reported levels of technology access 
and use increased. 

To gain a greater understanding of the association between students’ reported technology access and 
use and mathematics achievement, we used FILM to predict math achievement for each of the three 
Student Access and Use elements (Laptop Access Days, Core-Content Learning, and Florne Learning). 
Results in Table 7.8, si mi lar to TAKS reading outcomes, show that the extent to which students 
reported using their laptops for Flome Learning was a statistically significant predictor of TAKS 
mathematics scores. The Flome Learning effect on mathematics achievement was similarly positive for 
Cohort 1 (0.68 T- score point), Cohort 2 (0.51 T-score point), and Cohort 3 (0.50 T-score point). As an 
example, after controlling for the other variables, an economically advantaged, non-minority, male 
eighth grader with a Flome Learning score about one standard deviation above average (z, = 1.08), had 
a 0.68 T-scorc point higher TAKS mathematics score. As the extent of laptop use for Flome Learning 
increased, mathematics achievement increased incrementally. 


Table 7.8. Hierarchical Regression Models Predicting the Effects of Implementation Components 
(Including Elements of Student Access and Use) on TAKS Mathematics Achievement 


Predictor 

Cohort 1 
Eighth Graders 

Cohort 2 
Seventh Graders 

Cohort 3 
Sixth Graders 

Gamma 

Coefficient 

r-value 

Gamma 

Coefficient 

r-value 

Gamma 

Coefficient 

r-value 

Intercept 

51.612 

85.74*** 

48.787 

85.55*** 

47.912 

79.65*** 

Teacher-level predictors 

School poverty 

8.304 

2.82** 

4.617 

1.61 

5.191 

1.49 

Immersion Support 

1.432 

3.06** 

0.703 

1.09 

-0.186 

-0.24 

Classroom Immersion 

-0.244 

-0.34 

0.612 

1.80 + 

0.340 

0.70 

Student-level predictors 

Spring 2006 T score 

0.699 

27 . 99 *** 

0.728 

27.24*** 

0.686 

31 42 *** 

Laptop Access Days 

- 0.002 

- 0.01 

0.244 

1.81 

0.278 

1.02 

Core-Content Learning 

0.019 

0.07 

0.032 

0.15 

0.057 

0.29 

Home Learning 

0.675 

3.52** 

0.508 

2.54* 

0.504 

2 . 68 ** 

Female 

-0.490 

-1.54 

-0.215 

-0.69 

0.974 

2.49* 

African American 

-1.061 

- 1.68 

-0.539 

-1.08 

-1.169 

-1.27 

Hispanic 

-0.562 

-0.78 

-1.249 

-2.42* 

-0.647 

-0.94 

Eco. Disadvantaged 

-0.801 

- 2 . 12 * 

0.140 

0.33 

-0.506 

-0.91 


*p < .05. **p < .01. ***p < .001. 

Note. Numbers of mathematics teachers: Cohort 1 = 39, Cohort 2 = 40, and Cohort 3 = 33. 
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In contrast to Home Learning, students’ reported number of Laptop Access Days and the frequency of 
laptop use for Core-Content Learning in classrooms were typically positive but non-significant 
predictors of TAKS mathematics achievement across the three student cohorts. 

Conclusions 

In this chapter we described factors associated with the implementation of technology immersion and 
the relationships between the implementation of immersion components and students’ academic 
achievement. Key findings are the following. 

• School administrators appeared to advance implementation through their provision of supports 
for teachers’ technology immersion efforts. Teachers’ opinions of the strength of 
administrative leadership for technology were significantly associated with their perceived 
levels of implementation support (i.e., collective support for technology innovation, parent and 
community support, the prevalence of technical support, and the robustness of professional 
development). 

• Although associations between school characteristics and implementation indicators were 
often statistically insignificant, overall trends showed that schools with larger student 
enrollments tended to have slightly lower implementation levels than schools with fewer 
students, and schools with higher percentages of economically disadvantaged students tended 
to have lower implementation levels. Technical support was a significant problem at larger 
schools, whereas collective teacher support for technology innovation was a significant issue 
for schools with greater proportions of disadvantaged students. In contrast, the schools’ 
achievement context (percentage of students passing all TAKS tests), was positively 
associated with nearly all of the implementation indicators. 

• Teachers’ views on the robustness of campus Professional Development showed the strongest 
relationships with the elements of Classroom Immersion and the composite index. 

Surprisingly, teachers’ average implementation levels for the Classroom Immersion Index and 
its elements were negatively associated with the schools’ TAKS achievement. 

• The school level of Student Access and Use (of technology) was significantly associated with 
teachers’ overall support for technology innovation and the quality of technical support that 
addressed infrastructure and maintenance issues causing barriers to students’ laptop use. 
Student Access and Use was negatively associated with larger school size and the percentages 
of minority and economically disadvantaged students. 

• Data analyses for individual students and their teachers showed that the campus measure of 
Immersion Support and reading and mathematics teachers’ reported levels of Classroom 
Immersion were inconsistent predictors of students’ TAKS reading and mathematics 
achievement. 

• Conversely, the level of Student Access and Use (of technology) was a statistically significant 
positive predictor of students’ TAKS reading achievement for Cohort 1 (eighth graders), 
Cohort 2 (seventh graders), and Cohort 3 (sixth graders). Of the three elements of Student 
Access and Use, students’ use of their laptops for Home Learning — a measure of the extent to 
which a student uses a laptop outside of school for homework in the four core-subject areas 
and for learning games — was the strongest predictor of TAKS reading achievement. 

• Si mi lar to TAKS reading, the level of Student Access and Use was a statistically significant 
positive predictor of students’ TAKS mathematics achievement for each of the three student 
cohorts. Like reading, students’ use of their laptops for Home Learning was the strongest 
implementation predictor of TAKS mathematics achievement. 
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8. Conclusions and Implications 


The third-year evaluation describes schools’ progress in creating technology immersed environments and 
provides cumulative results on the effects of technology immersion on teachers and students (i.e., a laptop 
computer for every student and teacher, wireless access throughout the campus, curricular and assessment 
resources, professional development, and ongoing technical and pedagogical support). Additionally in the 
third year, we assessed longer term effects of immersion on students’ academic achievement and explored 
associations between project implementation and student outcomes. Although this report concentrates on 
information gathered during the third project year (2006-07), analyses also include data from the first 
(2004-05) and second (2005-06) years. 

Our research design is quasi-experimental, with 42 grades 6 to 8 middle schools drawn from rural, 
suburban, and urban locations in Texas. Schools are divided equally between the treatment group (21) and 
control group (21). The middle schools are typically small (402 students, on average); however, 
enrollments vary widely (from 83 to 1,447 students). While schools are mainly concentrated in small or 
very small Texas districts (less than 3,000 students), about a third of schools are in very large districts 
(10,000 or more students). 

The study focused on three student cohorts in the third year. Cohort 1 included eighth graders (2,586 
treatment, 2,863 control) who completed their third project year, Cohort 2 included seventh graders 
(2,644 treatment, 2,882 control) who finished their second project year, and Cohort 3 included sixth 
graders (2,597 treatment, 2,840 control) who concluded their first year. Students in the cohorts were 
predominantly minority (65%) and economically disadvantaged (67%). 

Study Limitations 

The sample selection process and matching procedures used with the quasi-experimental design appear to 
have produced a sample of schools with good internal validity, in that there are no large, statistically 
significant treatment-control group differences. Although baseline data confirmed that the comparison 
groups were reasonably well matched, we have used statistical methods to adjust for differences that 
could have arisen from sampling variability. A threat to internal validity was introduced in the third year 
when control schools began to plan for technology immersion and most of the control teachers received 
laptops, instructional resources, and more intensive professional development. Consequently, the 
magnitude of the third-year treatment effects may be underestimated, especially for teachers. 

Generalization of findings to a broader population (external validity) is a primary study limitation. 
Compared to Texas middle-school students as a whole, students in the sample schools arc substantially 
more Hispanic and less White and African American. Middle schools are also smaller than the statewide 
average (402 students versus 667). Schools also are located either in small or very small districts (64%) or 
large districts (36%), which differs from the statewide distribution of schools. Additionally, for many 
variables, the study relies on self-reported data from surveys of teachers and students — thus, some 
findings on changes in proficiencies and practices reflect respondents’ perceptions. Nonetheless, the 
triangulation of evidence from multiple sources (surveys, classroom observations, state demographic and 
test databases, multiple student cohorts) verifies the robustness of findings. Researchers are confident that 
reported effects can be attributed to the treatment. 


89 




Major Third-Year Findings 


Outcomes, as described below, represent the effects of technology immersion for schools that generally 
had less than full implementation levels. Although the quality of schools’ implementation improved 
slightly in the third year, we estimate that just a quarter of middle schools (5) achieved substantial 
immersion levels, while the remaining schools (16) had minimal to partial immersion. Major findings 
from the third year arc described in the following sections. 

Effects of Technology Immersion on Teachers and Teaching 

In the third project year we assessed the effects of immersion on teachers and teaching by examining 
teachers’ rates of growth on mediating variables across four time points (fall 2004 and spring 2005, 2006, 
and 2007). Analyses involved 1,684 teachers, including 816 in immersion schools and 868 in control 
schools. Even though control teachers benefited in the third year from initial steps toward implementation 
of the technology immersion model, we found that being part of a technology immersion school affected 
teachers in a number of ways. 

Immersion teachers continued to grow in technology proficiency and in their use of technology for 
professional productivity at significantly faster rates than control teachers. Although both treatment 
and control teachers made progress toward meeting the state’s Technology Application Standards, 
immersion teachers grew at a faster pace. Self-assessments of Technology Proficiency across four time 
points indicated that immersion teachers were increasingly more technology literate than control teachers 
in areas involving technology operations (e.g., sending email and using software applications) and 
pedagogical skills (e.g., creating electronic presentations and creating lessons plans integrating 
technology). Estimated yearly growth trajectories for immersion teachers in schools with average student 
poverty, compared to control, were more than twice as steep (0.31 and 0.13 scale-score points per year, 
respectively, on a 7-point scale). Similarly, teachers in immersion schools grew in their use of technology 
to enhance their Professional Productivity at a significantly faster rate than control teachers (0.20 and 0.08 
scale-score points per year, respectively, on a 5-point scale). Consequently, teachers in immersion schools 
used technology more frequently for puiposes such as communicating with students, posting information 
on a website, administering online assessments, and accessing model lesson plans. 

Teachers at schools with higher concentrations of student poverty grew in technology proficiency at 
a slower rate. Consistent with previous years, teachers who taught at schools with higher levels of 
student poverty grew in technology proficiency at significantly slower rates than their peers in more 
advantaged schools. As the level of school poverty increased, the proficiency gap between teachers 
widened. Weaker supports for implementation at higher poverty immersion schools may at least partially 
explain teachers’ slower progress. 

Teachers at immersion schools expressed increasingly stronger ideological associations across years 
with technology integration and learner-centered practices. Initially, immersion and control teachers 
expressed similar views on instructional practices involving technology, but immersion teachers altered 
their beliefs about practices at a significantly faster rate. For Technology Integration, the mean estimated 
growth for immersion teachers in schools with average poverty was 0.59 scale-score point per year 
compared to 0.24 for control teachers (on a 7-point scale). Thus, immersion teachers increasingly 
employed actions supporting curricular and instructional infusion of technology, such as promoting 
students’ authentic problem solving or critical thinking through technology. Immersion teachers also 
expressed increasingly stronger affiliations with constructivist or learner-centered practices, such as 
having students establish individual learning goals and emphasizing experiential learning. The estimated 
yearly growth rates in learner-centered practices for immersion and control teachers in average poverty 
schools were 0.38 and 0.20 scale-score points, respectively, on a 7-point scale. 
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Teachers at immersion schools had more collegial interactions on technology-related issues than 
control teachers, and students used technology more often in immersion classrooms. Teachers at 
immersion schools compared to control reported increasingly more frequent collaborative interactions 
with their colleagues that supported instructional practices involving technology (e.g., developing lesson 
plans or exchanging information about students). Also, immersion teachers increased the frequency of 
their students’ Classroom Activities involving technology at a more rapid pace than control teachers (for 
teachers in schools with average poverty, 0.23 scale-score point per year on a 5-point scale versus 0.03 
point). Although student activities with technology have steadily increased in immersion classrooms, 
third-year statistics indicted that students used various technology resources infrequently (i.e., about once 
or twice a month, M - 2.65). Mean statistics, however, obscured the substantial teacher-to-teacher 
variation in the frequency of students’ technology activities both across and within subject areas. Similar 
to previous years, English language aits, science, and social studies teachers had students use technology 
considerably more often than mathematics teachers. 

Cumulative evidence suggests that laptop computers and digital resources have allowed students in 
technology immersion schools to experience slightly more intellectually demanding work. New 

resources in technology immersion schools and classrooms are expected to promote students’ higher level 
thinking through more challenging and relevant learning activities that support academic achievement 
(e.g., Bransford et al., 2003; Newman & Associates, 1996; Newmann, Bryk, & Nagoaka, 2001). 
Accordingly, researchers have observed lessons in immersion and control teachers’ classrooms and rated 
the Intellectual Challenge of lessons (Newmann et al., 1995). Although observations of core-subject 
classes (English language arts/reading, mathematics, science, and social studies) in spring of 2005, 2006, 
and 2007 revealed no statistically significant differences between the overall Intellectual Challenge of 
immersion and control teachers’ instruction, the sizes of effects favoring immersion teachers increased 
across years. In particular, immersion teachers’ lessons had a greater emphasis on Higher Order Thinking 
over time compared to control teachers. Still, despite positive progress, results for all observed classrooms 
indicated that lessons in core classes generally failed to intellectually challenge middle-school students, 
with average ratings mostly below 2 on the 5-point scale. 

As control teachers experienced elements of technology immersion in the third year, the differences 
favoring treatment teachers for school-level variables began to dissipate. In contrast to the first two 
project years, there were no significant differences in the third year between treatment and control 
teachers’ perceptions of either administrative leadership for technology or technical support, and although 
treatment teachers reported significantly stronger teacher support for technology as well as parent and 
community support, the effect sizes were much smaller. (See data for comparisons in Appendix F.) Thus, 
as control teachers experienced components of technology immersion, si mi lar to treatment teachers, they 
began to view their schools’ technology environments as more supportive. 

Effects of Technology Immersion on Students and Learning 

In the third project year, we measured student mediating variables across four time periods for Cohort 1 
eighth graders (fall 2004 and spring 2005, 2006, and 2007), three periods for Cohort 2 seventh graders 
(fall 2005 and spring 2006 and 2007), and two periods for Cohort 3 sixth graders (fall 2006 and spring 
2007). Analyses for Cohorts 1, 2, and 3, respectively, included 1,337 immersion and 1,467 control 
students; 1,595 immersion and 1,671 control students; and 1,829 immersion and 2,164 control students. 
Controlling for important school and student characteristics, key findings include the following. 

Technology immersion significantly increased students’ technology proficiency and reduced the 
proficiency gap between economically advantaged and disadvantaged students. Across three cohorts, 
students in technology immersion schools have made greater progress in mastering the Texas Technology 
Applications standards than control students (e.g., sending an email attachment, creating a presentation, 


91 



managing documents, using spreadsheets, and keeping track of websites). For Cohort 1, technology 
immersion had a positive and enduring effect on the technology proficiencies of students from lower 
socioeconomic backgrounds. By the end of the third year, economically disadvantaged eighth graders in 
immersion schools were growing in proficiency at a significantly faster yearly rate (0.38 scale-score point 
on a 5-point scale) than either their more affluent immersion peers (0.31 point) or control-group students 
(about 0.28 point). For Cohort 2, both economically advantaged and disadvantaged immersion students 
grew in proficiency at faster rates (0.43 and 0.43 scale-score points, respectively) than their control-group 
counterparts (0.27 and 0.27 scale-score points). Similarly, for Cohort 3, immersion had a significantly 
positive effect on sixth graders’ technology proficiency (Effect Size [ES] = 0.30). 

Technology immersion significantly increased the frequency of students’ classroom technology use 
and their interactions with peers in small-group activities. Across three cohorts, students in immersion 
schools used technology applications significantly more often in their core-subject classrooms than 
control students. For Cohorts 1 and 2, the yearly growth rates in Classroom Activities for economically 
advantaged and disadvantaged immersion students ranged from 0.25 to 0.34 scale-score points (on a 5- 
point scale), compared to 0.04 to 0.10 points for comparable control-group students. Results for Cohort 3 
students, similarly, revealed significant and practically important differences in Classroom Activities 
favoring immersion schools (ES = 0.79). Despite significant increases, third-year statistics (similar to 
teachers’ reports) indicated that students used technology resources infrequently in core classes (about 
once or twice a month). 

Along with greater uses of classroom technology, students in immersion schools also had more frequent 
opportunities to learn in small groups with their classmates. Cohorts 1 and 2 immersion students had 
increasing opportunities for small-group work with their peers, whereas control students reported less 
frequent small-group activities as they advanced to higher grade levels. Cohort 3 students, likewise, had 
more opportunities for small-group interactions than control students (ES = 0.29). In general, as 
immersion teachers altered their beliefs about instructional practices, they began to configure student 
classroom activities differently. 

Students at immersion schools, compared to control, reported mounting technical problems over 
time when they used computers at school. Cohorts 1 and 2 immersion students reported increasing 
technical problems using computers across years compared to control students, with the growth in 
problems statistically significant for Cohort 1 (eighth graders). Cohort 3 students at immersion schools 
(sixth graders), who inherited laptops that had been used by students during two previous school years, 
also reported significantly more technical problems than control group- students. Although increased 
problems appeared to accompany aging laptops, mean scores in spring 2007 indicated that students, on 
average, rarely (a few times a year) or just sometimes (once or twice a month) had problems using 
computers at school. 

Technology immersion and control students regarded themselves as similarly self-directed learners. 

Since the independent and self-guided learning afforded through one-to-one technology was expected to 
positively affect students’ personal self-direction, students were asked to complete the Style of Learning 
Inventory as a measure of Self-Directed Learning. Findings in the third year replicated first- and second- 
year results showing there was no significant immersion effect on students’ self-direction. As both 
immersion and control students in Cohorts 1 and 2 progressed from lower to higher grade levels, their 
responses to statements measuring self-direction revealed significantly negative growth trends. Thus, 
students reported less self -regulated learning behaviors across time. Results for Cohort 3, sixth graders, 
similarly, revealed no significant immersion effect on students’ self-direction (ES = 0.1 1). 
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Students in immersion schools had significantly fewer disciplinary actions, similar levels of school 
satisfaction, and significantly lower school attendance rates than control-group students. 

Researchers have associated one-to-one computing with increased student engagement as measured by 
indicators such as stronger commitment to academic work, reduced discipline problems, and increased 
school attendance (e.g., MEPRI, 2003; Lowther, Ross, & Morrison, 2003; Rockman ET AL., 1998; 1999; 
Russell, Bebell, & Higgins, n.d.). Consistent results for the three student cohorts involved in our study 
show that immersion students exhibited significantly stronger school engagement through more positive 
behavior. However, they did not express greater satisfaction with school, and they attended school less 
regularly than control students. 

Behavior and discipline. Disciplinary Action Reports submitted to the Texas Education Agency (TEA) 
for each student during the 2006-07 school year, si mi lar to the previous two years, showed that 
immersion students had proportionately fewer behavioral and disciplinary problems than their 
counterparts in control schools. Cohorts 1, 2, and 3 immersion students had an average of 0.65, 0.53, and 
0.47 disciplinary actions, respectively, compared to 0.90, 0.86, and 0.75 for control students. Although 
the effect sizes for the mean differences were small, having fewer disciplinary actions per student in 
middle schools may have important practical benefits in terms of student learning time as well as day-to- 
day personnel time and effort required for addressing discipline problems that remove students from 
classrooms. 

School satisfaction. In the first project year, Cohort 1 students reported significantly higher school 
satisfaction than control students. However, for the second and third project years, there were no 
significant differences in school satisfaction between Cohorts 1, 2, and 3 students at immersion and 
control schools. Across both comparison groups, student cohorts expressed correspondingly modest levels 
of satisfaction with the kinds of work they do in classes and with the relevance of their schoolwork. First- 
year school satisfaction ratings reported by Cohort 1 students may have reflected the celebrations and 
initial excitement that students experienced when laptops were distributed for the first time. 

School attendance. Contrary to expectations, across three cohorts, students in immersion schools had 
significantly lower school attendance rates than control-group students. The yearly estimated negative 
change in attendance for Cohorts 1 and 2 students was greater for immersion students compared to 
control. Thus, at the end of eighth grade, advantaged students in immersion schools had an average 
attendance rate of 96.3% compared to 97.2% for control students, and at the end of seventh grade, 
advantaged students in immersion schools had an average attendance rate of 96.5% compared to 97.3% 
for control students. Economically disadvantaged immersion students, similarly, had significantly lower 
attendance rates than their control-group counterparts. Likewise, Cohort 3 immersion students attended 
school at a significantly lower rate than control students. 

The reason why immersion students attend school at a lower rate is unclear. It is possible that some 
students may occasionally skip school so that they can use their laptops at home. Surprisingly, as detailed 
in the section to follow, immersion students’ lower average school attendance was not always associated 
with lower academic achievement. This contrasts with other research linking lower school attendance 
rates with lower test scores (e.g., Shapley et al., 2004; Sheehan, 2006). 

Effects of Technology Immersion on Academic Achievement 

Increasing middle school students’ academic achievement in core subjects as measured by state 
assessments is the ultimate goal of technology immersion. For analyses reported below, students’ Texas 
Assessment of Knowledge and Skills (TAKS) scale scores were standardized and then normalized as T 
scores with a mean of 50 and a standard deviation of 10. Analyses for Cohort 1 included about 1,380 
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immersion and 1,600 control students. Cohort 2 included about 1,550 immersion and 1,725 control 
students, and Cohort 3 included about 1,780 immersion and 1,990 control students. 

Longitudinal data and data for multiple student cohorts allowed researchers to examine achievement 
effects over time. Given that small effects are noteworthy when effects are replicated (e.g„ Abelson, 1985; 
Cohen, 1994; Schmidt, 1996), we have reported effects as statistically significant using a less stringent 
criterion (p - .10) when findings provided evidence substantiating important trends. Texas students complete 
TAKS tests annually in reading and mathematics, so the evidence on student effects is stronger for those 
subject areas. In contrast, evidence for science, social studies, and writing is limited because students’ 
complete those assessments at periodic intervals. 

Technology immersion had no statistically significant effect on students’ TAKS reading 
achievement. After controlling for student and school poverty, there were no statistically significant 
effects of immersion on the TAKS reading growth rates for either Cohort 1 (eighth graders) or Cohort 2 
(seventh graders). The immersion effects were positive but not by significant margins. Across cohorts, 
economically disadvantaged students grew in reading achievement at significantly faster rates than their 
more affluent peers (0.38 and 0.52 T-score points per year for Cohorts 1 and 2 immersion students, 
respectively; 0.17 and 0.13 T-score points for control-group students, respectively). Thus, for 
economically disadvantaged immersion students, annual growth provided a substantial boost in reading 
achievement over time. For Cohort 3 sixth graders, after controls for students’ prior achievement, 
demographic characteristics, and school poverty, there was no statistically significant effect of immersion 
on students’ 2007 TAKS reading T scores. Si mi lar to the other cohorts, the immersion effect was positive 
but not by a significant extent (about 0.08 T-score point). 

Technology immersion had a statistically significant effect on TAKS mathematics achievement, 
particularly for economically advantaged and higher achieving students. After controlling for student 
and school poverty, technology immersion had a statistically significant effect on students’ TAKS 
mathematics growth rates for Cohorts 1 and 2 students. For Cohort 1, a significant interaction effect 
revealed that economically advantaged students in immersion schools increased their math achievement at 
a significantly faster rate than disadvantaged students (about 0.40 T-score point per year versus 0.02 T- 
score point, respectively), and at a faster rate than both economically advantaged and disadvantaged 
control-group students (-0.18 and -0.16 T-score points, respectively). For Cohort 2, economically 
advantaged and disadvantaged immersion students had TAKS mathematics growth rates (0.26 and 0.30 T- 
score points per year, respectively) that significantly outpaced their control-group counterparts (-0.44 and 
-0.40 T-score points). 

For Cohort 3 sixth graders, after controlling for students’ prior achievement, demographic characteristics, 
and school poverty, there was a statistically significant effect of immersion that acted through students’ 
pretest scores. Other factors being equal, as students’ TAKS pretest scores increased, the achievement gap 
favoring immersion students over control widened for 2007 TAKS mathematics scores. Thus, immersion 
had a stronger and significant effect for higher achieving sixth graders. 

Students who had greater access to laptops and used laptops for learning to a greater extent, 
especially outside of school, had significantly higher TAKS reading and mathematics scores. Given 
that the level of implementation of technology immersion varied from school to school, classroom to 
classroom, and student to student, we used a series of FILM models to investigate the relationships 
between implementation levels and student academic achievement. Specifically, Student Access and Use 
was an aggregate implementation measure of the extent to which a student had access to a laptop 
throughout the school year (number of days), the frequency of technology use for learning in core-subject 
classes, and the extent of laptop use for homework and learning games. Student-level FILM results 
showed that the composite measure of Student Access and Use was a statistically significant positive 
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predictor of students’ TAKS reading and mathematics achievement for each of the three student cohorts. 
Of the three elements of Student Access and Use, students’ use of their laptops for Home Learning — a 
measure of the extent to which students used laptops outside of school for homework in the four core- 
subject areas and for learning games — was the strongest predictor of both TAKS reading and mathematics 
achievement. In contrast, we found that reading and mathematics teachers’ reported levels of Classroom 
Immersion were typically insignificant predictors of students’ academic achievement. 

The findings for Student Access and Use, and especially, Home Learning, arc important because they 
reinforce the understanding that student achievement in school depends on individual student initiative as 
well as what happens outside of school. Results also highlight the important role that individual laptops 
play in promoting ubiquitous learning and in equalizing the out-of-school learning opportunities for 
students in disadvantaged family and school situations (Bransford, Brown, & Cocking, 2003; Burbules, 
2007; Dede, 2007). Mounting evidence suggests that technology immersion’s effect on student 
achievement is mediated through students’ greater ownership of learning allowed through the provision of 
technology resources that expand where and how learning occurs, that stimulate student attention and 
self-motivation, and that change the ways students’ acquire knowledge and skills (Shapley et al., 2007). 

Schools and teachers play a key role in creating conditions that enable students to use their laptops and 
new technologies to become more capable and dedicated learners. That is, students who attend schools 
that keep laptops in students’ hands and encourage the use of laptops at school and at home for academic 
pursuits, earn higher test scores. Moreover, students who have at least some teachers who start 
technology-infused lessons at school that either require or inspire students to continue working at home, 
or who teach lessons that arouse students’ interest in further learning, use their laptops for more academic 
purposes that promote higher achievement. Additional research is now needed to better understand 
exactly how laptops and particular technology resources contribute to student thinking, learning, and 
academic success. 

The effects of technology immersion on reading and mathematics achievement generally became 
stronger over time as teachers and students became more accomplished technology users. Across 
three project years, the immersion effects on reading and mathematics achievement evolved. In the first 
project year, the immersion effects on TAKS reading and mathematics scores were negative. In the 
second year, immersion effects were typically positive, but not by statistically significant margins. In the 
third year, significantly positive immersion effects on TAKS mathematics scores emerged for each of the 
three student cohorts. Additionally, links were established between higher levels of student 
implementation and achievement. These finding underscore the importance of longitudinal studies in 
assessing the impacts of educational initiatives on student achievement. Evidence from this study and 
others show that a greater number of project implementation years is associated with increasing effects on 
achievement outcomes (e.g., Borman et al., 2003; Borman, 2005). For technology immersion, higher and 
more consistent levels of implementation might also produce stronger immersion effects on student 
achievement. 

Evidence regarding the effects of technology immersion on students’ TAKS social studies, science, 
and writing achievement is inconclusive. Since TAKS tests for social studies, science, and writing are 
not administered annually, immersion effects for these subject areas cannot be replicated across cohorts 
and years. Accordingly, it is not possible to draw definitive conclusions about the effects of technology 
immersion for these subject areas. Available results typically show no statistically significant effects of 
immersion, with differences between groups favoring immersion students for TAKS social studies and 
control students for TAKS science and writing. 

Social studies. The TAKS social studies test is administered for the first time in 8th grade, so students’ 

7th grade TAKS reading scores were used to adjust for prior achievement. After controlling for Cohort 1 
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eighth graders’ reading achievement, demographic characteristics, and school poverty, there was no 
statistically significant effect of immersion on TAKS social studies scores. The immersion effect was 
positive (0.57 T-score point) but not by a significant degree. 

Science. After controlling for prior achievement (5th grade science score), demographic characteristics, 
and school poverty, there was no statistically significant effect of immersion on Cohort 1 eighth graders’ 
TAKS science achievement. The immersion effect was negative (-0.24 T-score point). Also, there was a 
statistically significant interaction related to students’ pretest scores. As TAKS pretest scores increased, 
the achievement gap favoring control students over immersion widened for 2007 TAKS science scores. 
Thus, there was a significantly negative effect on TAKS science scores for higher achieving eighth 
graders at immersion schools. 

Writing. After controlling for Cohort 2 seventh graders’ pretest writing scores (4th grade writing score), 
demographic characteristics, and campus poverty, there was no statistically significant effect of 
immersion on students’ 2007 TAKS writing scores. The immersion effect was negative (-0.28 T-score 
point). Similarly, there was no statistically significant immersion effect on the 2006 writing scores for 
Cohort 1 students who completed writing assessments during the second project year. The immersion 
effect, similar to Cohort 2, was negative (-0.91 T-score point). The testing mode for TAKS writing, 
however, may have influenced student outcomes because the TAKS assessment is administered in paper- 
and-pencil format. Some research studies show that traditional assessments may underestimate the writing 
performance of students who are accustomed to using word processors for writing because they are not 
allowed to use technology when being tested (Russell & Haney, 1997; Russell & Plati, 2001). 

Nature of Third-Year Implementation 

In the sections to follow, we describe how the varying levels of project implementation may have 
contributed to reported outcomes. 

Although the overall level of implementation increased between the second and third project years, 
just a quarter of schools reached substantial levels of technology immersion. Full implementation of 
the technology immersion model requires support in several ways: Leadership, Teacher Support (buy-in), 
Parent and Community Support, Technical Support, and Professional Development. Given adequate 
supports, teachers are expected to reach high levels of Classroom Immersion, and Student Access and Use 
of technology is expected to be robust. Mean immersion standard scores revealed small increases between 
the second and third implementation years for each of the immersion support components as well as for 
teachers’ overall level of Classroom Immersion. In contrast, the level of Student Access and Use was 
stable across years. 

Despite progress, evidence from immersion standard scores and the Implementation Index, a composite 
score measuring the overall presence of immersion components, indicated that just a quarter of middle 
schools (5) had a stronger presence of the immersion components that more nearly approximated 
expected implementation standards compared to the other schools (16). Third-year findings, however, 
showed that technology immersion could have positive effects on teachers and students even at lower 
implementation stages. 

School administrators advanced implementation through their provision of supports for teachers’ 
technology immersion efforts, whereas teachers’ greater support for immersion along with 
technical support elevated Student Access and Use. Teachers’ opinion of the strength of administrative 
leadership for technology at their school was significantly associated with their perceived levels of 
implementation support (i.e., collective support for technology innovation, parent and community 
support, the prevalence of technical support, and the robustness of professional development). 
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Additionally, teachers’ overall support for technology innovation and the extent to which teachers 
believed that the quality of technical support addressed infrastructure and maintenance issues causing 
barriers to students’ laptop use, were significantly associated with greater Student Access and Use. To 
reach higher levels of immersion, many schools needed stronger supports for implementation in the third 
year. 

Core-subject teachers at the majority of schools reported only partial levels of Classroom 
Immersion in the third year; teachers at a few schools, however, made collective progress in 
creating technology-immersed classrooms. Immersion standard scores for each of five elements of 
Classroom Immersion showed slightly stronger implementation in the third year, with the largest 
increases for teachers’ ideological affiliations with Technology Integration and Learner-Centered 
Instruction, and the smallest change for Student Activities with technology in the classroom. There were 
notable increases in teachers’ use of technology as a communication tool and for the enhancement of their 
own professional productivity. Core teachers (as a whole) at about a fifth of schools reached a substantial 
level of Classroom Immersion in the third year. 

HLM analyses for individual students and their teachers, surprisingly, showed that reading and 
mathematics teachers’ reported levels of Classroom Immersion, in most cases, were statistically 
insignificant predictors of students’ TAKS reading and mathematics achievement. Measurement issues, 
within classroom variability, and interdisciplinary effects provide potential explanations for findings. 

First, measures of the elements of Classroom Immersion may not include variables that link directly to 
student academic achievement. Second, teachers’ views reflected their overall perceptions of classroom 
technology, whereas the nature of classroom immersion varied for individual students due to their 
inconsistent access to laptops because of lost days for repairs, disciplinary issues, or other reasons. 
Individual students also utilized their laptops within classes in different ways. Consequently, student- 
reported activities with technology provided the most salient predictors of academic effects. 

Additionally, immersion effects on academic achievement may reflect the efforts of multiple teachers 
rather than single core-subject teachers. For example, the use of laptops for social studies research and 
compositions may positively affect reading outcomes. Similarly, exposure to investigations and problem 
solving activities in science may positively affect mathematics scores. Additional analyses could 
contribute to a better understanding of the relationship between teachers’ use of technology and students’ 
academic outcomes. 

Students’ access to and use of laptops for learning within and outside of school generally fell short 
of substantial to full implementation. Students at more than two-thirds of schools had just partial levels 
of Student Access and Use in the third year, whereas students at about a third of schools had only 
minimal access and use. Students’ opportunities to use their laptops both within classrooms and outside of 
school were affected by the number of days that students actually had their laptops. Year-to-year 
comparisons indicated that Laptop Access Days declined between the second and third project years. In 
contrast, students reported small increases in the third year in their use of laptops for Core-Content 
Learning and Home Learning. 

Larger schools and schools with a greater proportion of economically disadvantaged students had 
lower levels of implementation. Overall trends showed that schools with larger student enrollments 
tended to have slightly lower implementation levels than schools with fewer students, and schools with 
higher percentages of economically disadvantaged students tended to have lower implementation levels. 
Technical support was a significant problem at larger schools, whereas collective teacher support for 
technology innovation was a significant issue for schools with greater proportions of disadvantaged 
students. Teachers at higher poverty schools also grew in technology proficiency at significantly lower 
rates, and student access to and use of technology decreased as the percentages of minority and 
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economically disadvantaged students in a school increased. On the contrary, the schools’ achievement 
context (percentage of students passing all TAKS tests), was positively associated with nearly all of the 
implementation indicators. 

These findings arc consistent with other studies showing that unequal technology opportunities between 
higher and lower socioeconomic status schools generally persist despite the infusion of resources that 
have diminished the digital divide (Education Week, 2007; Warschauer, 2007). Clearly, if students are to 
realize the full potential of laptops and technology resources, larger schools and schools serving 
disadvantaged student populations must have adequate supports for technology immersion in place to 
meet the specific needs of the school’s teachers, students, and parents prior to implementing an 
immersion project. 
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Appendix A 

Theoretical Framework for Technology Immersion — Literature Review 


The theoretical framework (Figure 1.1) guides the evaluation. The research literature underpinning the 
framework is provided in sections to follow for school, teacher, and student variables. In some cases, 
sources relate specifically to educational technology, whereas in other instances, evidence comes from 
studies of education in general. Research evidence for some valuables is relatively robust; in other areas, 
evidence is weaker. Although research on one-to-one computing initiatives has grown in recent years, 
there arc still few experimental studies or studies with well matched comparison groups that provide 
evidence of causal effects. 

School-Level Variables 

In a “technology immersed” school, technology resources are ingrained in the school’s organizational and 
cultural environment. Technology immersion, therefore, should change not just classroom instruction and 
learning, but also the nature of interactions between student and teacher, teacher and teacher, teacher and 
principal, and the school within the surrounding community (Dwyer, 1994). Considering the systemic 
nature of technology immersion, the evaluation examines factors that help to explain how and under what 
conditions technology affects students’ learning opportunities and academic achievement. The sections 
below describe the key variables of interest at the school level, including leadership, innovative culture, 
parent and community support, and technical support. 

Leadership 

Over the past several decades, researchers have concluded consistently that school leadership is critical in 
developing and maintaining conditions that support school change and academic improvement (e.g., 
Flallinger & Fleck, 1996 cited in Spillane, 2003; Leithwood, Seashore, Anderson, & Wahlstrom, 2004). 
Similarly, administrative support is a major factor that influences technology integration (International 
Society for Technology in Education, 2002; Bradburn & Osborne, 2007). Leaders in a technology- 
enhanced environment must be “champions of technology, teaching, learning, and students” (Johnston 
and Cooley, 2000, p. 95). The principal, in particular, is a pivotal figure in effective technology 
implementation. The visionary principal is one who sees the integral relationship between technology and 
education, and marshals resources to help teachers master effective practices (Tinucci, 2000). 
Additionally, effective principals are “transformational leaders” who create more collaborative teaching 
and learning environments through their facilitation of opportunities for technology specialists and 
teachers to share their knowledge, experiences, and insights (Bradburn & Osborne, 2007). 

A consistent vision and plan for change is also essential for whole-school reform efforts such as 
technology immersion. Shared vision, or buy-in, moves schools toward substantive changes in 
instructional approaches and improved student outcomes (Leithwood et al., 2004). Conversely, without 
broad-based support, technology immersion may be untapped resource that has little impact on student 
learning (Cradler, 1992; Means & Olson, 1994). 

Innovative Culture 

The school culture may either promote or impede whole-school initiatives such as technology immersion. 
When undertaking innovation, the organization’s shared commitment to change and ability to build 
capacity for doing things in a new way are important (Senge, 1999). In education, some schools are more 
successful than others in enacting and sustaining innovation, and in more effective schools, changed 
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practice is a collective rather than an individual enterprise (Fullan, 1993). Similarly, movement towards 
new ways of teaching and learning with technology is more significant if teachers are able to work 
collaboratively (Chapman, 1996). Shared professional learning opportunities provide a viable means to 
stimulate innovative teaching practices (Birman, Desimone, Porter, & Garet, 2000; Dibbon, 2003). 
Considering prior research, we believe that educators’ collective experiences at immersion campuses will 
advance their shared understanding of technology’s use and encourage integration efforts. Schools that 
begin the project with more collaborative cultures may advance at a faster pace (Fullan, 1999). 

Parent and Community Support 

The local community also may influence technology immersion. Its constituents consist of parents, 
neighborhood residents, local professionals, and elected school board officials. Educating and involving 
the community has been identified as a key component in ensuring successful change in educational 
practices (Desimone, 2002; Goertz, Floden, & O’Day, 1996; Leithwood et al., 2004). If parents and 
community members are “on the same page” as the school with regard to technology immersion, they can 
contribute the kind of supports and resources required for changes in educational practices. At immersion 
campuses, community outreach may take many forms, such as participation on a technology committee, 
attendance at informational sessions or workshops, the dissemination of information through district and 
campus websites, or media releases to spread the word about technology immersion. Most important, in a 
one-to-one computing project, parents must be partners in assuming responsibility for the appropriate use 
of laptops outside of the school. 

Technical Support 

Texas has strongly supported the infusion of technology into its schools (Texas Education Agency, 2002; 
2006). Consequently, at the start of this project, both treatment and control campuses had existing 
inventories of technology hardware, software, and educational programs. Districts and campuses also had 
human resources such as technology coordinators and technical support personnel who supported 
technology at the district and campus levels. Given existing contextual conditions, and the infusion of 
resources through technology immersion, an examination of the nature and quality of technical support at 
participating schools is important. 

Teacher Variables 

At the teacher level, we theorize that technology immersion leads to increased technology proficiency, 
greater use of technology for professional productivity, more frequent opportunities for students to use 
technology in classrooms, and pedagogical changes such as increased technology integration and more 
learner-centered instruction. New technology also is expected to advance the intellectual demands of 
lessons and assignments. Moreover, teachers in schools that are immersed in technology should begin to 
collaborate more often with their peers as they experiment with new instructional technologies and digital 
resources. 

Technology Proficiency 

A number of studies associate teachers’ technology proficiencies with technology implementation. 
Research indicates that teachers need a solid foundation of technology literacy before they can 
successfully integrate technology into the curriculum. Teachers must learn to use technology comfortably 
and efficiently (Dusick, 1998-1999; Goldsworthy, 2000). Studies also show that teachers with stronger 
computer skills use technology in a greater number of ways and on a more regular basis, and these 
teachers are more likely to increase their technology-use frequency over time (Ronnkvist, Dexter, & 
Anderson, 2000). Moreover, teachers with the strongest technology proficiencies use technology in more 
innovative ways in their content areas (Becker, 2000). 
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Unfortunately, research indicates that many teachers lack the proficiencies and understanding necessary 
to apply technology resources to instruction and learning effectively. A national study found that more 
than half of teachers felt only somewhat prepared to use technology for instruction, and more experienced 
teachers felt less prepared than their more novice counterparts (Smerdon et al., 2000). Surveys of Texas 
teachers have revealed improvements in proficiencies across time, but teachers’ proficiency levels 
remained below targeted standards (Shapley, Benner, Heikes, & Pieper, 2002). Similarly, 2005-06 
statewide outcomes for the Texas Teacher STaR Chart (a measure of teachers’ technology readiness) 
showed that nearly three of four Texas teachers rated their progress relative to the Teaching and Learning 
area as either Early Tech (14.7%) or Developing Tech (55.6%). Only one in four teachers believed they 
had attained proficiencies designated as Advanced Tech (23.7%) or Targeted Tech (5.8%) (Texas Region 
10 Education Service Center & Texas Education Agency, 2006). 

Professional Productivity 

Skilled teachers also are more likely to use technology as a tool to enhance their own professional 
productivity, including actions such as communicating with students and parents by email, creating 
electronic lesson plans, or accessing information from the Internet for lessons (Shapley et al., 2002). 
Researchers typically have not investigated teachers’ use of technology for professional productivity, but 
it is important in Texas because state standards call for teachers to use technology for communicating 
effectively, as well as for acquiring, analyzing, and evaluating a variety of electronic information. In an 
immersed school, teachers arc expected to increasingly communicate by email, report attendance and 
submit lesson plans electronically, post information on a class or campus website, and analyze and 
interpret electronic data from assessments. 

Classroom Technology Use 

The link between increased technology access and increased classroom use is well documented. Teachers 
use computers and the Internet more often when technologies are available in their classrooms rather than 
in other locations in the school (Becker, 2001; Smerdon et al., 2000). Teachers involved in Maine’s one- 
to-one initiative, in fact, used technology more often, possessed a broad knowledge of technology 
resources, and made progress in incorporating technology into practice (MEPRI, 2004). Thus, we assume 
that providing laptops for each student in an immersed school will increase students’ opportunities for 
classroom technology use. 

Technology Integration and Learner-Centered Instruction 

Abundant technology hardware and software is important, but if those resources arc not well integrated 
into instructional approaches and learning experiences, the impact on student achievement may be 
negligible. Notably, studies show that teachers’ ideologies affect the likelihood of technology integration, 
with teachers’ perceived costs and benefits influencing changed practices (Zhao & Frank, 2003). 

Research also suggests that teachers’ understanding of new learning theories and understanding of how 
technology supports enriched learning opportunities are important (Bransford, Brown, & Cocking, 2003; 
Johnston & Cooley, 2001). Researchers studying the Apple Classrooms of Tomorrow (ACOT) found that 
abundant access to classroom technology changed teachers’ beliefs as well as their instructional approach. 
Teachers’ beliefs and practices evolved along a technology integration continuum that gradually led to 
effective instructional practices. Movement from the entry phase to invention (technology-intensive 
environments) required time and ongoing support (Dwyer, Ringstaff, & Sandholtz, 1991). 

Specifically, researchers found that ACOT teachers began to incorporate more collaborative work and 
fewer teacher-centered, lecture-oriented lessons in favor of student-centered ones (Baker, Gearhart, & 
Herman, 1994). Subsequent studies, likewise, have found evidence of teachers adjusting their pedagogical 
style, with students taking more responsibility for their own learning in one-to-one laptop classrooms 
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(MEPRI, 2003), and classroom structures that shifted from large group to students working independently 
or to more student-centered activities (Rockman ET AL., 1998; Russell, Bebell, Cowan, & Corbelli, 
2002). Other evidence, however, suggests that some teachers view technology as an add-on or reward for 
students who finish their seatwork rather than an integral paid of their pedagogical repertoire (Rockman 
ET AL., 1998). 

Intellectual Challenge 

Technology immersion’s main benefit may stem from opportunities for more complex modes of teaching 
and learning. Research on technology-infused classrooms reveals positive attributes, such as the ability to 
bring real-life problems into the classroom or high-quality simulations of them. Technology also allows 
teachers to model thinking strategies and allows individual learners to approach tasks in different ways 
using different learning strategies (Goldman, Cole, & Syer, 1999; Many, Fyfe, Lewis, & Mitchell, 1996; 
Sulla, 1999; Temple & Rodero, 1995). This view of technology’s potential for more advanced learning 
contrasts with evidence on prevailing classroom conditions. While three-quarters of teachers nationally 
report using computers or the Internet for instruction, most lessons fail to involve complex inquiries, 
explorations, or problem-solving activities (Doherty & Orlofsky, 2001). Similarly, Texas students and 
teachers use technology mainly at a basic level, with technology used most often for tasks such as 
conducting Internet research on an assigned topic (Shapley et al., 2002). 

Collaboration 

Research suggests that teachers need time to discuss technology use with other teachers. Professional 
collaboration includes communicating with educators in similar situations and with teachers who have 
previous technology experiences. Collaboration may occur in face-to-face meetings or through 
technology venues such as email or videoconferencing. Teachers in the Maine laptop initiative, for 
example, believed their most effective professional development activity was informal help from 
colleagues. E-mail, listservs, and websites enabled Maine teachers to exchange information and stay in 
touch with their peers (MEPRI, 2003). Moreover, Zhao and Frank report that “teachers who perceived 
pressure from colleagues were more likely to use computers for their own puiposes, and teachers who 
received help from colleagues were more likely to use computers with their students” (2003, p. 825). 

Student Variables 

Over the past decade, a growing body of research points to positive effects of technology on students’ 
skills, learning, and achievement. In the research literature, evidence suggests that technology access 
fosters positive student effects for technology use, technical proficiencies, motivation and engagement, 
intellectually challenging schoolwork, self-direction, and to a lesser extent, academic achievement. 

Technology Use 

Technology is used more often for instructional and learning puiposes in one-to-one laptop classrooms 
(Russell, Bebell, & Higgins, n.d.). Additionally, students involved in ubiquitous technology initiatives use 
technology more often outside of school. Russell et al. (n.d.) found that students in one-to-one classrooms 
used computers at home more frequently for academic puiposes. Likewise, other researchers found that 
students spent less time watching television and more time on homework after they received laptop 
computers (Baldwin, 1999). Moreover, laptops provided a means of “closing the digital divide” between 
more advantaged students who had access to computers and the Internet at home and those without 
technology outside of school (Rockman, 2003). 
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Technology Proficiency 

Students’ technology proficiencies reportedly increase with ubiquitous technology. Laptop students in 
one study considered themselves more proficient users of Word, Excel, PowerPoint, the Internet, email, 
and CD-ROMS than non-laptop students (Rockman ET AL., 1998). Similarly, fifth and sixth graders who 
received laptop computers in another study reported increased computer skills and better Internet research 
capabilities (Lowther, Ross, & Morrison, 2001). In another study, German high school students with 
laptops made greater gains than comparison students on measures of technology literacy, such as 
knowledge of hardware and the operating system, productivity tools, and Internet use (Schaumburg, 
2001). 

Motivation and Engagement 

Numerous studies report links between one-to-one technology and increased student engagement 
(MEPRI, 2003; Rockman ET AL., 1998; Russell et al., n.d.; Woodul, Vitale, & Scott, 2000). The five- 
year ACOT evaluation established a link between technology use and student attitudes. Students 
voluntarily used time outside of school to work on technology-based projects, and they often initiated 
their own computer-related projects (Baker, Gearhart, & Herman, 1994). Students involved in the Maine 
Learning Technology Initiative, similarly, found school and learning more interesting and preferred using 
laptops for most school-related tasks (MEPRI, 2003). 

Additionally, studies have examined the relationship between technology and student behavior. In a 
statewide study in Florida, middle schools experienced fewer student conduct violations and disciplinary 
actions as the number of computers in use per student increased (Barron, Hogarty, Kromery, & Lenkway, 
1999). Other studies, likewise, report decreased discipline problems associated with one-to-one 
computing (Baldwin, 1999; MEPRI, 2003). In another study, a computerized curriculum positively 
affected the psychosocial and academic outcomes of students identified as chronically disruptive (Aeby, 
Powell, & Carpenter- Aeby, 1999-2000). 

An evaluation of the North Carolina Laptop Notebook Project revealed a strong correlation between 
computer use and improved school attendance. Students participating in the laptop program had fewer 
absences and late arrivals as compared to non-participants (Stevenson, 1998). In Henrico County Public 
Schools in Virginia, preliminary evidence linked increased student motivation, engagement, and interest 
to one-to-one computing (Zucker & McGee, 2005). 

Intellectual Work 

Existing studies suggest that student technology use most commonly involves productivity tools, Internet 
research, and drill and practice activities. Activities involving higher-order thinking and peer 
collaboration, such as technology-based projects, multimedia authoring, problem solving with 
spreadsheets or databases, or correspondence with experts, are less common (Becker, 1999, 2001; 

Denton, Davis, & Strader, 2001; Smerdon et al., 2000). Contrary to prevalent practice, some believe that 
technology, at its best, can “facilitate deep exploration and integration of information, high-level thinking, 
and profound engagement by allowing students to design, explore, experiment, access information, and 
model complex phenomena” (Goldman et al., 1999). Additionally, technology allows students increased 
access to and use of a wide range of information, facilitating greater inquiry and investigation, exposure 
to places and resources beyond the classroom, and development of a stronger knowledge base (CEO 
Forum, 2001; Johnston & Cooley, 2001). 

New circumstances and opportunities — not technology on its own — can impact student achievement. 
Several studies have established tentative links between interactive technologies and higher level 
reasoning and problem solving (Baker et al., 1994; Hopson, Simms, & Knezek, 2002). New technologies, 
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apparently, allow students to build knowledge by doing, receiving feedback, and continually refining their 
understanding (Barron et al., 1999; Bereiter & Scardamalia, 1993). Technology also provides a medium 
for bringing real-world problems into the classroom for students to explore and solve. Students involved 
in the Jasper Woodbury Problem Solving Series, for example, had positive gains in mathematical problem 
solving, communication abilities, and attitudes toward mathematics (e.g.. Cognition and Technology 
Group at Vanderbilt, 1997). 

Self-Directed Learning 

Several studies associate technology use with increased student self-directed learning. The connection 
assumes that working one-to-one with technology allows students to have hands-on, self-directed 
experiences since they work independently much of the time. The theory of self-regulation posits that a 
learner who knows how to be self-directed and independent will be more successful than one who is 
highly dependent on structured guidance (Zimmerman, 1989). The teacher’s role is to scaffold learning by 
making thinking processes more tangible and by modeling learning strategies (Bolhuis, 1996; Corno, 

1992; Leal, 1993). Since self-directed learners are responsible owners and managers of their own learning 
process, control shifts over time from teachers to learners (Garrison, 1997). 

Self-regulated or self-directed strategics enable learners to solve problems in new domains (Ertmer & 
Newby, 1996; Morrow, Sharkey, & Firestone, 1993) or to solve real-world problems (Bolhuis, 1996; 
Temple & Rodero, 1995). For example, in computer-supported science classes, middle-school students 
took more responsibility for their learning, and concurrently, displayed greater competence in complex 
problem-solving strategics (Raghavan, Sartor is, & Glaser, 1997). Another study suggested that students 
who learned in a self-directed environment were more productive. When writers were allowed to choose 
their own topics, they wrote more often and they wrote longer pieces (Morrow et al., 1993). 

Academic Achievement 

The ultimate goal of technology immersion is increasing the academic progress of students. Available 
evidence on the effects of laptops on student achievement comes from a few studies that have made 
comparisons between student groups with and without technology. Findings, although limited, have 
generally been positive. 

The strongest evidence on the effects of laptops on achievement is in the area of writing. Lowther, Ross, 
and Morrison (2001, 2003) reported highly significant effects favoring sixth- and seventh-grade students 
with laptops over control students for dimensions of writing, such as ideas and content, organization, and 
style. In a less methodologically rigorous study, Rockman ET AL. (1999) found that laptop students 
outscored non-laptop students on four measures of writing, including content; organization; language, 
voice, and style; and mechanics, conventions, and presentation. 

Some studies also have reported positive effects of one-to-one laptop access on students higher order 
problem solving (Lowther et al., 2003). Evaluation of a laptop project in Beaufort County, West Virginia, 
which focused on outcomes measured by a nationally standardized achievement test, found that laptop 
students participating in the program for two years had higher language, reading, and mathematics scores 
than non-laptop students (Stevenson, 1998). However, since there was no statistical control for prior 
achievement, findings are in doubt. Certainly, additional research studies with experimental designs are 
needed to draw definitive conclusions about the effects of one-to-one initiatives on student achievement. 
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Appendix B 

Characteristics of Participating Schools 


The schools participating in the study are compared in Table B.l. The distribution of middle schools 
across campus and district enrollment categories shows the comparability of treatment and control groups 
For both groups, middle schools are typically small (enrolling 600 students or less), and they are located 
either in small or very small districts (enrolling 2,999 students or less) or large districts (enrolling 10,000 
students or more). 


Table B.l. Campus and District Enrollment by Comparison Group 



Immersion N= 21 

Control /V=2 1 

Number of students 

Number 

Percent 

Number 

Percent 

Campus 

300 or less 

12 

57.1 

12 

57.1 

301-600 

5 

23.8 

4 

19.0 

601 or more 

4 

19.0 

5 

23.8 

District 

999 or less 

8 

38.1 

8 

38.1 

1,000-2,999 

6 

28.6 

5 

23.8 

3,000-9,999 

0 

0.0 

0 

0.0 

10,000 or more 

7 

33.3 

8 

38.1 


Note. Two campuses (one experimental and one control) were excluded from the 
comparison groups in the second year. 


Tables B.2 and B.3 provide campus-level data for each of the 42 schools included in the study. Again, 
data show that the treatment and control schools are reasonably well matched on baseline characteristics. 
Middle schools are highly concentrated in rural and very small districts across the state. Still, over a third 
of districts and schools are in large cities or suburban locations in or around cities. The sample also 
includes campus charter schools (one each for the treatment and control group) located in a major urban 
district. 
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Table B.2. Characteristics of Technology Immersion and Matched Control Schools 
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Table B.3. Characteristics of Technology Immersion and Matched Control Schools ( Continued ) 
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Appendix C 

Survey Items and Scale Reliabilities 


Table C.l. Items and Reliabilities for School-Level Scales 


Scale/ 

Item 

Fall 

2004 

Cronbach s Alpha 

Spring. Spring 
2005 2006 

Spring 

2007 

Leadership and System Support 

0.91 

0.92 

0.94 

0.97 

The principal consults with staff before making decisions about instructional technology 
that affect us. 





In this school, there are clear expectations that technology will be used to enhance student 
learning. 





The principal in my school actively encourages teachers to pursue professional 
development geared towards curricular integration of technology. 





Our school has a well-developed technology plan that guides all technology integration 
efforts. 





The principal is an effective leader for instructional technology in this school. 





Overall, considering the uses of technology in my school today, 1 am confident that this use 
is leading to increased student achievement. 





The principal encourages teachers to be innovative and try new methods. 





The principal is willing to support through funding or manpower teachers' efforts at 
technology integration. 





Administrators in this school help teachers to use technology to access, analyze, and 
interpret student performance data. 





Teachers receive adequate administrative support to integrate technology into classroom 
practice. 





Teachers and administrators rely on research-proven teaching and learning principles in 
making decisions about technology use. 





When our school has professional development focused on technology, the principal often 
participates. 





Classroom Technology Integration 

0.67 

0.68 

0.76 

0.75 

Students have adequate access to technology resources in my classroom (e.g., digital 
cameras, scanners, projectors). 





1 incorporate the TEKS for student technology applications into my content-area lessons. 





1 have received sufficient training to incorporate technology into my instruction. 





1 use technology to assess student performance and plan instruction. 





Technical Support 

0.71 

0.71 

0.66 

0.67 

Most of our school computers are kept in good working condition. 





Internet connections in my class are often too slow or not working. 





My requests for technical assistance are addressed in a timely manner. 





Materials (e.g., software, printer supplies) for classroom use of computers are readily 
available in my school. 





Problems such as computers freezing or an inability to access the Internet make it difficult 
for me to use technology. 





Innovative Culture 

0.78 

0.79 

0.82 

0.82 

Teachers in this school share an understanding about how technology will be used to 
enhance learning. 





Teachers in this school are continually learning and seeking new ideas. 





Teachers are not afraid to learn about new technologies and use them with their class(es). 





Teachers in this school are generally supportive of technology integration efforts. 





Parent and Community Support 

0.78 

0.79 

0.85 

0.84 

Parents support our school's emphasis on technology. 





The surrounding community actively supports our instructional efforts with technology. 
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Table C.2 Items and Reliabilities for Teacher-Level Scales 


Scale/ 

Item 

Fall 

2004 

Cronbach’s Alpha 

Spring Spring 
2005 2006 

Spring 

2007 

Technology Proficiency: 1 am confident that 1 can... 

0.97 

0.97 

0.97 

0.97 

Send email to coworkers, parents, or peers. 





Collaborate through subscribing to a discussion list. 





Create an address book to send email to several people at once. 





Send a document as an attachment to an email message. 





Use a variety of search strategies, including key word and Boolean logic to find Web 
pages related to my subject matter interests. 





Search for and find a Web site with information about the Alamo. 





Create my own World Wide Web home page. 





Keep track of Web sites 1 have visited so that 1 can return to them later. (An example is 
using bookmarks.) 





Find primary sources of information on the Internet that 1 can use in my teaching. 





Use a spreadsheet (e.g., excel) to enter and calculate numbers. 





Use a spreadsheet to create a pie chart. 





Create a newsletter using desktop publishing techniques, including graphics & text in 3 
columns. 





Perform basic software application functions such as opening an application program 
and creating, modifying, printing, and saving documents. 





Plan, create, and edit documents using word processing software (e.g., Word). 





Use the computer to create a slideshow presentation (e.g., Powerpoint). 





Plan, create, and edit databases using database software (e.g., Access). 





Use a database to search for and sort information and create reports. 





Use graphic organizers and/or systems thinking software (Inspiration, Stella, etc.) to 
teach concepts. 





Use drawing or painting software (e.g., Paint, Illustrator) to create pictures. 





Create a lesson or unit that incorporates subject matter software as an integral part. 





Use technology to collaborate with other colleagues who are distant from my 
classroom. 





Describe 5 software programs that 1 would select and use in my teaching. 





Write a plan with a budget to buy technology for my classroom. 





Teach my students about copyright issues as they relate to the Internet including citing 
sources. 





Take photos with a digital camera, save in a digitized format, and use in an electronic 
document. 





Scan images from a print source such as a book, save them in a digitized format, and 
use them in an electronic document. 





Create products incorporating text, audio, video, and graphics using multimedia 
authoring programs (e.g., Authorware, HyperStudio). 





Professional Productivity: As a teacher, 1... 

0.93 

0.94 

0.94 

0.91 

Keep administrative records (e.g., attendance). 





Manage student assessment data (e.g., electronic gradebooks). 





Use technology to analyze and interpret student data to guide instruction. 





Create electronic lesson plans. 





Communicate with students. 





Communicate with parents. 





Communicate with colleagues/other professionals. 





Create instructional materials (e.g., tests, handouts). 





Gather information from the internet to create a lesson (e.g., text, video, clipart). 





Access model lesson plans integrating technology. 





Deliver information using presentation software (e.g., Powerpoint). 





Deliver information using multimedia presentations (text, audio, video, grgraphics). 





Post homework, class requirements, or project information on a website. 





Administer a formative assessment using Texas Mathematics Diagnostic System. 
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Scale/ 

Item 

Fall 

2004 

Cronbach’s Alpha 

Spring Spring 
2005 2006 

Spring 

2007 

Administer other online assessments. 





Use the internet at home for instructional purposes. 





Use a computer to do schoolwork at home. 





Students’ Technology Use: Students in my class use technology to... 

0.95 

0.98 

0.98 

0.96 

Express themselves in writing (e.g., word processing). 





Learn and practice skills (e.g., instructional software or educational games). 





Enter, calculate, and graph information (e.g., Excel spreadsheet). 





Create a database of information for a class project (e.g., Filemaker Pro, Access). 





Create and make presentations (e.g., Powerpoint). 





Communicate by email with peers, experts, or others on topics they are studying. 





Use online discussions to gather information for an assignment (e.g., through 
discussion boards or videoconferencing). 





Conduct internet research on an assigned topic. 





Conduct multimedia research (reference CDs, online encyclopedias). 





Enhance or express conceptual understanding through simulation/modeling software. 





Visually represent or investigate concepts (e.g., through concept mapping, graphing, 
reading charts). 





Produce print products (e.g., desktop publishing). 





Produce multimedia reports/projects (e.g., with video, graphics, and sound editing). 





Analyze information using tools such as graphing calculators or digital microscopes. 





Design web sites or web pages. 





Complete a test or quiz (e.g., online assessments, Texas Math Diagnostic System). 





Other (specify) 





Collaboration: As a teacher, 1... 

0.90 

0.92 

0.93 

0.92 

Act as a coach or mentor to other teachers or staff at my school. (May include teaching 
in-service workshop in your school.) 





Receive coaching or mentoring from an external (non-school) source such as a 
professional curriculum developer. 





Receive coaching or mentoring from an internal source, such as another teacher or 
technology coordinator. 





Have informal discussions with colleagues regarding strategies for integrating 
technology. 





Receive feedback from other teachers based on their observations of my teaching. 





Provide feedback to other teachers based on my observations of their teaching. 





Consult with other teachers about certain students' technology skills or use. 





Exchange feedback with other teachers based on student work that used technology. 





Work with a subject-area peer to develop a lesson plan or class activity using 
technology. 





Work with a colleague in a different subject area to develop a lesson plan. 





Participate in a study group with other teachers on a technology-related topic. 





Technology Integration 

0.94 

0.95 

0.95 

0.91 

1 alter my instructional use of the classroom computer(s) based upon the newest 
software applications and research on teaching, learning, and standards-based 
curriculum. 





My students discover innovative ways to use classroom computers to make a 
difference in their lives. 





1 allocate time for students to practice their computer skills on the classroom 
computer(s). 





1 integrate the most current research on teaching and learning when using the 
classroom computer(s). 





In my classroom, students use technology-based computer and Internet resources 
beyond the school (NASA, other government agencies, private sector) to solve 
authentic problems. 





My students’ authentic problem solving is supported by continuous access to a vast 
array of computer-based tools and technology. 





1 plan computer-related activities in my classroom that will improve my students’ basic 
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Scale/ 

Item 

Fall 

2004 

Cronbach’s Alpha 

Spring Spring 
2005 2006 

Spring 

2007 

skills (e.g., reading, writing, math computation). 





It is easy for me to design student-centered, integrated curriculum units that use the 
classroom computer(s) in a seamless fashion. 





1 seek out activities that promote increased problem-solving and critical thinking using 
the classroom computer(s). 





Using cutting edge technology and computers, 1 have stretched the instructional 
computing in my classroom. 





Learner-Centered Instruction 

0.75 

0.80 

0.81 

0.81 

Students' authentic use of information and inquiry skills guides the type of instructional 
materials used in my classroom. 





My students are involved in establishing individual goals within the classroom 
curriculum. 





In addition to traditional assessments, 1 consistently provide alternative assessment 
opportunities that encourage students to “showcase” their content understanding in 
nontraditional ways. 





My instructional approach emphasizes experiential learning, student involvement, and 
students solving “real-world” issues. 





Resistance to Integration 

0.70 

0.72 

0.77 

0.81 

1 do not find computers to be a necessary part of classroom instruction. 





Using the classroom computer(s) is not a priority for me this school year. 





1 do not find the use of computers to be practical for my students. 
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Table C.3. Items and Reliabilities for Student-Level Scales 



Cronbach’s Alpha 1 

Scale/ 

Item 

Fall 

2004 

Spring 

2005 

Spring 

2006 

Spring 

2007 

Technology Proficiency: How far along are you in learning to... 

0.94 

0.94 

0.94 

0.94 

open, create, modify, print, and save documents 





use a digital camera and/or scanner to get pictures into the computer 





send a document as an attachment to an email 





keep track of Web sites 1 have visited so that 1 can return to them later (using 
bookmarks, etc.) 





enter information on the computer using proper keyboarding skills 





gather information from CD-ROMS 





use online reference databases (online encyclopedias, newspapers, Library of 
Congress, etc.) to gather information 





use a search engine to find information about a topic (Alamo, etc.) on the Web 





narrow Web searches using key words and Boolean logic (such as “or,” “and,” or “not”) 





use online discussions with experts or mentors to gather information 





evaluate information found on the Web for accuracy 





use a word processor (AppleWorks, Word, etc.) to write and print a story or report 





use a spreadsheet (AppleWorks, Excel, etc.) to enter and calculate numbers 





use a spreadsheet to create graphs 





use a database (AppleWorks, Access, etc.) to enter information 





use a database to search for and sort information and create reports 





use software (Keynote, PowerPoint, etc.) to create a presentation 





use drawing or painting software (Paint, Illustrator, etc.) to create pictures 





use a video camera to make a video 





use software (HyperStudio, Authorware, etc.) to create a multimedia product 





use email to send and receive messages 





use software (Frontpage, Publisher, etc.) to create web pages 





Technology Use in School: In your English language arts, mathematics, social studies, 
and science classes, how often do your teachers have you... 

0.90 

0.92 

0.91 

0.92 

use a word processor (AppleWorks, Word, etc.) to write a story or report. 





use software to learn and practice skills (Riverdeep, Compass Learning, PLATO 
Learning, etc.). 





use a spreadsheet (Excel, etc.) to enter and calculate numbers or create graphs for an 
assignment. 





create a database of information (Filemaker Pro, Access, etc.) for a class project. 





create a presentation (PowerPoint, etc.) and present information to classmates or 
others. 





communicate by email with friends, experts, and others about topics you are studying. 





use online discussions to gather information for an assignment (discussion boards, 
videoconferencing, etc.). 





conduct Internet research on an assigned topic. 





use tools, such as graphing calculators or digital microscopes, to analyze information. 





produce print products (with desktop publishing software). 





create multimedia reports or projects (with video, graphics, and sound editing). 





use technology to complete a test or quiz. 





Other 





Technical Problems 

0.83 

0.85 

0.84 

0.77 

The computer is broken or slow. 





The program 1 need is not on the computer. 





The Internet connection is too slow or not working. 





A website 1 need is blocked by a filter. 





Sharing a computer makes it hard to finish assignments. 





My teacher can't fix things when something goes wrong. 





Other (describe) 
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Scale/ 

Item 

Fall 

2004 

Cronbach’s Alpha 

Spring Spring 
2005 2006 

Spring 

2007 

Small-Group Work: When students work together in small groups in my classes, we... 

0.80 

0.83 

0.83 

0.83 

review and give advice on each other’s work. 





tutor or coach each other on difficult work. 





make a presentation for the rest of the class. 





brainstorm solutions to problems. 





discuss previous class assignments. 





produce a report or project. 





School Satisfaction 

0.77 

0.82 

0.80 

0.80 

1 am satisfied with the work that 1 do in my classes. 





1 understand why 1 am doing the things we do in my classes. 





The things we do in my classes will help me as an adult. 





The work we do in my classes will be useful to me in the job 1 hope to have as an adult. 





1 work hard in my classes because the work is meaningful. 





What 1 learn in my classes is more important than the grade 1 receive. 





Self-Directed Learning 

0.88 

0.89 

0.89 

NA 

If I’m confused in class, 1 ask the teacher or another student for help. 





Sometimes, if 1 think an assignment is too tough, 1 purposely don't try hard. Then if 1 
don't do well, 1 don't feel bad. 





At the end of a project or assignment, I'll think about how hard 1 worked and whether 1 
would do anything differently next time. 





It's important to me that 1 understand my schoolwork really well. 





Even when 1 think my schoolwork is boring, 1 keep working until I'm finished. 





Before 1 begin studying, 1 think about or list the things I'm going to do during my study 
time. 





Even when I’m supposed to learn about something boring, 1 keep working until 1 finish. 





When my teacher writes comments on assignments, 1 don't read them unless 1 have to. 





When we start a new unit, 1 like to know what we're going to be learning and how I'll 
know if I’ve learned it well. 





When the teacher calls on me, and 1 make a mistake in class, 1 can honestly say that 1 
don't feel bad. 





When 1 do well on a big project, it's because I've worked hard. 





1 work harder than 1 need to on my schoolwork, because that’s just the way 1 am. 





I'll recopy my notes or make diagrams of what we're learning to try and remember it 
better. 





1 don't like asking for help with my schoolwork. 





If a topic is too hard, it's really hard for me to stay motivated. 





If 1 know I'm going to do badly on a task, 1 try to avoid it, even if 1 know I'd learn a lot 
from it. 





There are some subjects I'm just bad at. 





A lot of times, I'll wait until the last minute to do my homework or study for a test. 





1 know 1 can make a schedule to get my work done on time and stick to it. 





When I'm doing homework, 1 rush to finish if 1 have ,a friend coming over or if a good 
TV show is about to start. 





I'll look through mistakes 1 made on earlier assignments so 1 don't make the same 
mistakes on new assignments. 





When I'm done writing a report, 1 read it over carefully and think 0 about whether I've 
done a good job. 





Even if 1 try, 1 can't make myself concentrate on schoolwork when there are more 
interesting things to do. 





When I'm reading a chapter, 1 ask myself questions to make sure 1 understand the 
material. 





There are some subjects 1 just can't understand, even if 1 try hard. 





When 1 get a bad grade, 1 feel dumb. 





I'll pick a tough project where 1 would learn a lot over an easy project, even if it means 
I'll have to work harder to get a good grade 
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Scale/ 

Item 

Fall 

2004 

Cronbach’s Alpha 

Spring Spring 
2005 2006 

Spring 

2007 

This happens to me a lot: I'll study for a test and think 1 understand everything; then 1 
take the test and don't do very well. 





1 don't really take notes when I'm reading something for school. 





When 1 get a grade 1 don't like, I'll spend time trying to figure out what 1 could have 
done differently. 





When 1 do badly on a project, 1 feel okay as long as 1 did better than some of the other 
kids in my class. 





When 1 answer a question wrong in class, 1 end up wishing I'd never spoken up. 





When 1 get a bad grade, it's because 1 could have studied more or because 1 should 
have done something differently, like taking better notes. 





If I'm having trouble concentrating, 1 find a place to study where 1 won't be distracted. 





The things we're learning in my class are usually really interesting. 





If 1 have to choose, I'd rather get good grades in a class than learn a lot. 





When a big project or report is assigned, 1 make a mental or written schedule to make 
sure everything gets done on time. 





I'll usually ask someone (like my parents, friends or teacher) to give me feedback on 
my ideas when I'm working on a big assignment. 





1 know from past experience exactly what 1 have to do (like schedule a certain amount 
of time, or take notes in a particular way) if 1 want to do well on my schoolwork. 





If an assignment isn't going to count toward my grade, 1 don't need to know how well 1 
did on it. 





1 only feel bad about a low grade if 1 think 1 didn't work hard enough, or if 1 think 1 made 
careless mistakes 





When 1 read, 1 put the important ideas into my own words. 





When I'm not feeling motivated, 1 can't, make myself study. 





When 1 don't understand things in class, 1 end up thinking it's because I'm not that 
smart. 





When we have a reading assignment, I'll read through it one time, but 1 don't really go 
back through it to check how well 1 remember it. 





1 know 1 can do well in school if 1 try hard enough. 





1 don't ask for help, even if 1 don't understand the directions for an assignment. 





1 wouldn't do any homework if 1 didn't have to. 






123 





Appendix D 

Measurement of Implementation Fidelity 


Defining Technology Immersion 

The Texas Education Agency selected three lead vendors as providers of technology immersion packages 
(Dell Computer, Inc., Apple Computer Inc., and Region 1 Education Service Center [ESC]). Sections to 
follow provide descriptions of the components of technology immersion packages. 

Wireless Laptops and Productivity Software 

All vendors offered a wireless laptop as the mobile computing device. Campuses could select either 
Apple laptops (iBook and MAC OSX) or Dell laptops (Inspiron or Latitude with Windows OS). For 
Apple laptops, AppieWorks provides a suite of productivity tools, including Keynote presentation 
software, Internet Explorer, Apple Mail, iCal calendars, iChat instant messaging, and iLife Digital Media 
Suite (iMovie, iPhoto, iTunes, GarageBand, and iDVD). For Dell laptops, Microsoft Office includes 
Word, Excel, Outlook, PowerPoint, and Access. In addition, eChcilk serves as a “portal” to other web- 
based applications and resources included in the immersion package and a student-safe email solution. 
Region 1 ESC provided Dell products. 

Online Instructional and Assessment Resources 

Immersion packages included a variety of digital resources. Apple included the following online 
resources: netTrekker (an academic Internet search engine). Beyond Books from Apex Learning (reading, 
science, and social studies online), ClcissTools Math from Apex Learning (complete math instruction), 
ExploreLeaming Math and Science (supplemental math/science curriculum), TeenBizfOOO from Achieve 
3000 (differentiated reading instruction), and My Access Writing from Vantage Learning (support for 
writing proficiency). Dell, Inc. selected netTrekker (an academic Internet search engine) and Connected 
Tech from Classroom Connect (technology-based lessons and projects). Region 1 ESC selected 
Connected Tech but also added a variety of teaching and learning resources including Unitedstreaming 
(digital videos). Encyclopedia Britannica , EBSCO (databases), NewsBank, and K12 Teaching and 
Learning Center. For the Apple package, AssessmentMaster (Renaissance Learning) provides a formative 
assessment in all four core subject areas. Both the Dell and Region 1 ESC packages provide i-Know (CTB 
McGraw Hill) for core-subject assessment. In addition, all campuses have access to the online Texas 
Mathematics Diagnostic System (TMDS) and Texas Science Diagnostic System (TMDS) that are 
provided free of charge by the state. 

Professional Development 

Each immersion package includes a different professional development provider. Apple uses its own 
professional development model, whereas the Dell package relies on Pearson Achievement Solutions, a 
commercial provider (formerly Co-nect ), to support professional development. Region 1 ESC uses a 
combination of service center support plus other services offered through Connected Coaching and 
Connected University. Although the professional development models and providers differ, they all were 
expected to include some common required elements, such as support for immersion package 
components, the design of technology-enhanced learning environments and experiences, lesson 
development in the core-subject areas, sustained learning opportunities, and ongoing coaching and 
support. Individual districts and campuses collaborated with vendors to develop specific professional 
development plans for their teachers and other staff. 
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Technical and Pedagogical Support 

Each technology immersion package provider also is required to provide campus-based technical support 
to advance the effective use of technology for teaching and learning. Apple designed a Master Service 
and Support Program. Dell established a Call Center dedicated to technical support for TIP grantees as 
well as an 800 telephone number for hardware and software support. Region 1 ESC had an online and 
telephone HelpDesk to answer questions and provide assistance. 

In sum, the RFQ process created technology immersion packages with common elements. Still, the 
complexity and variability of the treatment makes it critically important for researchers to document not 
only how and how well technology immersion is implemented but also to identify factors that contribute 
to implementation variations. 

Measuring Implementation 

In the third year, we employed a two-part approach to the measurement of implementation fidelity. First, 
we used indicators to describe each campus’ progress on a 4-step scale toward immersion standards. 
Rating scales for components and related elements identified four levels of immersion: minimal (0 to 
1.99), partial (2.00 to 2.99), substantial (3.00 to 3.49), and full (3.50 to 4.00). Second, we used 
quantitative implementation indices that gauged the level of technology immersion using standardized 
scores (z scores). Both the immersion standard scores and implementation indices were derived from 
values for seven components: (a) Leadership, (b) Teacher Support, (c) Parent and Community Support, 

(d) Technical Support, (e) Professional Development, (f) Classroom Immersion, and (g) Student Access 
and Use. The following sections describe the seven components of technology immersion and related 
measurement procedures. Table D.l shows the scoring rubrics for immersion indicators, and Table D.2 
describes the data sources used to generate scores. 

Supports for Implementation 

Leadership. Our measure of administrative leadership comes from teacher survey items (12) that yield a 
Leadership scale score. Items assess the extent to which administrators involved staff in decisions, set 
clear expectations for technology use, encourage and participate in professional development, have a 
well-developed technology plan, promote teacher innovation, and provide necessary resources and 
administrative support. Teachers rated the extent of their agreement on a 5-point scale ranging from 0 
(, strongly disagree ) to 4 ( strongly agree). To achieve substantial to full immersion, teachers had to agree 
or strongly agree that administrators provided technology leadership. A Leadership Index was generated 
by transforming the scale score to a z score. 

Teacher Support. Although implementation may be affected by the characteristics of individual teachers, 
it also may reflect the collective disposition of teachers toward the adoption of new and innovative 
practices. Our measure of teacher commitment to technology immersion comes from teacher survey items 
(4) measuring a Teacher Support scale (i.e., Innovative Culture). Items gauged the extent to which 
teachers in the school share an understanding about technology use for student learning, are continually 
learning and seeking new ideas, are not afraid to learn about and use new technologies, and are generally 
supportive of technology integration efforts. Teachers rated the extent of their agreement on a 5-point 
scale ranging from 0 ( strongly disagree) to 4 ( strongly agree), with substantial to full immersion tied to 
the strength of teacher agreement. A Teacher Support Index was generated by transforming the scale 
score to a z score. 

Parent and Community Support. Support from parents and community members is also a key paid of 
implementation because they must understand the goals of technology immersion, assume responsibility 
along with their children, and assist in enacting effective policies. Our measure of Parent and Community 
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Support is a scale score composed of teacher survey items (2). These items indicate the extent to which 
parents support the school’ s emphasis on technology and the community actively supports instructional 
efforts with technology. Teachers rated the extent of their agreement on a 5-point scale ranging from 0 
(, strongly disagree ) to 4 ( strongly agree). Substantial to full immersion reflected the strength of teacher 
agreement. A Parent/Community Support Index was generated by transforming the scale score to a z 
score. 

Technical Support. On a fully immersed campus, sufficient technical support and a healthy 
infrastructure are expected to alleviate technical problems that might interfere with the use of technology 
in the classroom, school, and beyond. Our measure for technical support comes from teacher survey items 
(5) contributing to a Technical Support scale score. Teachers indicated the extent of their agreement on a 
5-point scale ranging from 0 ( strongly disagree) to 4 ( strongly agree) that computers arc kept in good 
working order, requests for assistance are addressed in a timely way, Internet connections work 
adequately, and classroom materials are readily available. A Technical Support Index was generated by 
transforming the scale score to a z score. 

Professional Development. In constructing measures of professional development, we drew from 
research conducted on the effectiveness of the Eisenhower Professional Development Program (e.g., 
Garet, Porter, Desimone, Birman, & Yoon, 2001). Key features of quality professional development 
provided a framework for examining dimensions of schools’ and vendors’ professional development 
models. Data for measures come from core-subject teachers’ responses to survey items. 

First, we measured the total number of Contact Hours that core-subject teachers spent in technology- 
related professional development during the past school year. In addition, professional development 
models for technology immersion were required to include a classroom support component, so we 
measured Classroom Support as the extent to which core teachers indicated that they received modeling, 
coaching or mentoring from an internal source (such as another teacher or technology coordinator), or an 
external source (such a professional curriculum developer). Teachers rated the frequency of support on a 
4-point scale linked to standards: 0 {never), 1.33 ( rarely — a few times a year), 2.67 {sometimes — once or 
twice a month), and 4 {often — once or twice a week or almost daily). 

To examine the Content Focus of teachers’ activities, we asked each teacher who participated in 
technology -related professional development to indicate the degree of emphasis the activity placed on 
curriculum, instructional methods, and lesson development in their core-subject area. Teachers’ responses 
were coded on a 5-point scale with 0 = no emphasis, 2 = minor emphasis, and 4 -major emphasis. As a 
measure of professional development Coherence, each core teacher who attended technology-related 
events indicated the extent to which the activity was consistent with the their goals for professional 
development, was based explicitly on what the teacher had learned in earlier professional development 
experiences, was followed up with activities that built on what the teacher learned in the professional 
development activity, was aligned with state or district standards and curriculum frameworks and with 
state and district assessments. To measure this indicator, teachers used a 5-point scale ranging from 0 {not 
at all) to 4 {to a great extent). A Professional Development Index was generated by averaging z scores for 
each of the four professional development elements. 

Extent of Implementation 

Classroom Immersion. The technology immersion packages included a variety of instructional and 
assessment resources designed to extend, supplement, or enhance core-subject teaching and learning. 
Wireless laptops, for example, were loaded with productivity software (i.e., either Appleworks or 
Microsoft Office) for students to use as a learning tool. Teachers and students also received a variety of 
digital resources and formative assessments to support content-area instruction and learning activities. 
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Indicators for Classroom Immersion, accordingly, assessed the extent to which core-subject teachers at 
immersion campuses utilized resources and embraced practices consistent with the technology immersion 
model. Classroom Immersion is measured by five elements: Technology Integration, Learner-Centered 
Instruction, Student Classroom Activities, Communication, and Professional Productivity. Measures of 
Technology Integration (10 items) and Learner-Centered Instruction (4 items) arc scale scores adapted 
from the Levels of Technology Implementation (LoTi) Questionnaire. Core teachers indicated the extent 
to which statements related to Technology Integration (e.g., I alter my instructional practices to support 
higher order thinking through technology) and Learner-Centered Instruction (e.g., I have students use 
information and inquiry skills) are true on a 5-point scale, including 0 ( not true of me now), 1 to 3 
(, somewhat true of me now), and 4 (very true of me now). 

Because teachers influence students’ classroom opportunities to use technology for learning academic 
content, we also used items from teacher surveys as a way to assess the extent to which teachers had 
students use various technology applications in core-subject classrooms (Student Classroom Activities). 
For example, survey items gauged how often students’ used a word processor to write a story or used 
software to learn and practice skills. Teachers’ responses were converted to a 5-point scale tied to 
immersion standards. Responses indicated how often students’ in a typical class used technology in 
particular - ways: 0 (never), 1.33 ( rarely — a few times a year), 2.67 (sometimes — once or twice a month), 
4.00 (often — once or twice a week — or almost daily). 

Teachers at immersion schools also are expected to use technology as a communication tool. 
Communication that advances student learning involves sending email to students, parents, or colleagues, 
or posting information and assignments on a class or school website. Technology also provides a way to 
improve teachers’ Professional Productivity, including the use of technology for purposes such as keeping 
records, analyzing data, developing lessons, or delivering information. Scale scores for Communication (4 
items) and Professional Productivity (11 items) are comprised of teacher responses on a 5-point scale 
indicating the frequency of activities: 0 (never) to 4.00 (almost daily). The Classroom Immersion Index 
was generated by averaging z scores for each of the five elements described above. 

Student Access and Use. This indicator gauged the extent of student access to laptop computers as well 
as the frequency of students’ laptop use for learning in core-content classrooms and at home. Three 
elements — Laptop Access Days, Core-Content Learning, and Home Learning — contribute to the 
component score. First, in an immersion school, students are expected to have access to wireless laptops 
for the entire school year - . Our measure of Laptop Access was calculated as the number of days out of the 
180-day school year that students actually had laptops available for use. Information for the indicator 
comes from an analysis of student survey items in which students indicated whether the school provided a 
laptop for student use, and if provided, how many days the laptop had been taken away (e.g., for misuse, 
misbehavior, failure to complete assignments, bad grades, or repairs). Student access scores, which could 
range from 0 days (no laptop) to 180 days (laptop available the full school year), were converted to the 0- 
4.00 continuous scale to measure progress toward the immersion standard. A Laptop Access Index was 
generated by transforming the continuous score to a z score. 

The potential for laptops to affect achievement depends largely on students’ opportunities to use 
technology for learning core academic content. Consequently, we used items from student surveys (4) to 
assess the frequency with which students used technology resources in their English/language arts, 
mathematics, science, and social studies classrooms (Core-Content Learning). Students’ responses were 
converted to a 4-point frequency scale tied to standards: 0 (never or rarely — a few times a year), 1.33 
(sometimes — once or twice a month), 2.67 (often — once or twice a week), and 4 (almost daily). A Core- 
Content Learning Index was generated by transforming the scale score to a z score. 
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Additionally, on a fully immersed campus, students should have access to their wireless laptops for 
learning both within and outside of school. Information for the measure of Home Learning comes from 
student survey items in which students indicated whether the school provided a laptop for student use, 
how often the student could take a laptop home, and if a laptop could be taken home, how often it was 
used for homework in core subjects or for learning games. A student’s use of the laptop for home learning 
was rated on a 6-point scale: 0 (no access to laptop outside of school), 1 ( restricted or full access to 
laptop outside of school), plus up to 5 additional points if a student used their laptop for homework in 
ELA , math, science, or social studies, or for learning games. Students’ scores were converted to the 0- 
4.00 scale as a measure of progress toward immersion standards, and a z score was generated. We 
generated the Student Access and Use Index by averaging z scores for each of the three elements 
described above. 
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Table D.2. Data Sources for Technology Immersion Implementation Indicators 
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Table D.2. Data Sources for Technology Immersion Implementation Indicators ( Continued ) 
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Table D.2. Data Sources for Technology Immersion Implementation Indicators ( Continued ) 
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Table D.2. Data Sources for Technology Immersion Implementation Indicators ( Continued ) 
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Appendix E 

Technical Appendix — Hierarchical Linear Modeling (HLM) 


Effects of Technology Immersion on Teachers and Teaching (Chapter 4) 

Researchers estimated the effects of immersion on teacher mediating variables using three-level 
hierarchical linear growth models. In our models, we posit that school poverty is related to teachers’ 
initial status and yearly growth rate. Statistical details are provided in Tables E.l, E.2, and E.3. The 
models’ simplicity aids in the interpretation of effects. More complex models, controlling for teacher 
demographic characteristics (gender, ethnicity, experience), described subsequently in Tables E.4, E.5 
and E.6, estimated nearly identical immersion growth coefficients. 


Table E.l. Descriptive Statistics for Teacher Variables: HLM Models with School Poverty 


Variable Name 

N 

Mean 

SD 

Repeated Measures Descriptive Statistics (Level 1) 

Survey Time 

6,736 

1.50 

1.12 

Technology Proficiency 

4,395 

4.87 

1.41 

Professional Productivity 

4,357 

3.23 

0.72 

Technology Integration 

4,146 

3.66 

1.56 

Learner-Centered Instruction 

4,289 

4.10 

1.37 

Resistance to Integration 

4,316 

2.32 

1.35 

Student Classroom Activities 

4,326 

2.14 

0.80 

Collaboration 

4,356 

2.51 

0.77 

School-Level Descriptive Statistics (Level 3) 

Immersion status (1 = yes, 0 = no) 

42 

0.50 

0.51 

School percent economically disadvantaged 

42 

69.83 

17.05 


Table E.2. Immersion (Fixed) Effect Analyses of Teacher Mediating Variables: 
HLM Models with School Poverty 


School-Level Scale 

School-Level 

Analysis 

Gamma 

Coefficient 

Standard 

Error 

t 

Technology Proficiency 


Initial status (fall 2004) 

4.673 

0.086 

54.61*** 


Immersion dummy 

-0.184 

0.121 

-1.52 


School poverty 

- 0.001 

0.333 

-0.40 


Growth rate 

0.134 

0.021 

6.35*** 


Immersion dummy 

0.176 

0.031 

^ 72*** 


School poverty 

- 0.002 

0.096 

-2.44* 

Professional Productivity 


Initial status (fall 2004) 

3.023 

0.061 

49 . 94 *** 


Immersion dummy 

-0.094 

0.083 

-1.14 


School poverty 

0.000 

0.226 

0.12 


Growth rate 

0.084 

0.010 

8.16*** 


Immersion dummy 

0.117 

0.019 

^ ^ 7 *** 


School poverty 

- 0.001 

0.064 

- 0.88 


Continued 
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Table E.2. Immersion (Fixed) Effect Analysis of Teacher Mediating Variables (Continued) 


Technology Integration 


Initial status (fall 2004) 

2.885 

0.069 

41 52*** 


Immersion dummy 2 

0.298 

0.103 

2.89** 


School poverty 

0.008 

0.288 

2.72* 


Growth rate 

0.237 

0.033 

7 24*** 


Immersion dummy 2 

0.348 

0.050 

6.96*** 


School poverty 

-0.003 

0.168 

-1.76f 

Learner-Centered Instruction 


Initial status (fall 2004) 

3.670 

0.060 

60.79*** 


Immersion dummy 

-0.028 

0.088 

-0.32 


School poverty 

0.006 

0.225 

2.55* 


Growth rate 

0.198 

0.025 

7 79 *** 


Immersion dummy 

0.186 

0.041 

4 54 *** 


School poverty 

- 0.002 

0.132 

-1.40 

Resistance to Integration 


Initial status (fall 2004) 

2.434 

0.048 

50.38*** 


Immersion dummy b 

-0.281 

0.065 

-4 34 *** 


School poverty 

-0.003 

0.159 

- 1 .72 J 


Growth rate 

0.032 

0.018 

1 .85* 


Immersion dummy b 

-0.017 

0.030 

-0.58 


School poverty 

0.001 

0.080 

1.12 

Student Classroom Activities 


Initial status (fall 2004) 

1.888 

0.047 

4Q 47 **** 


Immersion dummy 

0.075 

0.064 

1.17 


School poverty 

0.004 

0.167 

2.23* 


Growth rate 

0.034 

0.015 

2.36* 


Immersion dummy 

0.199 

0.027 

7 43 **** 


School poverty 

- 0.001 

0.083 

-0.91 

Collaboration 


Initial status (fall 2004) 

2.300 

0.051 

44 . 77 *** 


Immersion dummy 

0.107 

0.072 

1.49 


School poverty 

0.003 

0.171 

1.74f 


Growth rate 

0.035 

0.018 

1.92f 


Immersion dummy 

0.065 

0.029 

2 . 20 * 


School poverty 

0.000 

0.080 

0.06 


t p < .10; *p < .05; **p < .01; ***p < .001. 

d Immersion teachers had significantly higher initial technology integration scores. A latent variable regression, 
controlling for the effect of this initial difference on the growth rate, indicated that the difference between the 
original and adjusted immersion coefficients was not significant (the difference divided by the standard error 
of the difference = -.46). 

^Immersion teachers had significantly lower initial resistance to integration scores. A latent variable regression, 
controlling for the effect of this initial difference on the growth rate, indicated that the difference between the 
original and adjusted immersion coefficients was not significant (the difference divided by the standard error 
of the difference = -0.26). 
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Table E.3. Variance Decomposition from Conditional HLM Growth Models of Teacher 
Mediating Variables (with School Poverty) 


Scale/ 

Random Effect 

Variance 

Component 

df 

X 2 

P 

Technology Proficiency 

Level- 1 temporal variation 

0.3274 




Level-2 individual initial status 

1.7892 

1,227 

8,518.80 

0.000 

Level-2 individual growth rate 

0.0319 

1,227 

1,783.03 

0.000 

Level-2 school initial status 

0.0729 

39 

100.89 

0.000 

Level-2 school growth rate 

0.0041 

39 

83.60 

0.000 

Professional Productivity 

Level- 1 temporal variation 

0.1498 




Level-2 individual initial status 

0.3225 

1,221 

4,134.76 

0.000 

Level-2 individual growth rate 

0.0103 

1,221 

1,579.99 

0.000 

Level-2 school initial status 

0.0518 

39 

172.93 

0.000 

Level-2 school growth rate 

0.0012 

39 

73.55 

0.001 

Technology Integration 

Level- 1 temporal variation 

0.6988 




Level-2 individual initial status 

1.2833 

1,175 

3,500.47 

0.000 

Level-2 individual growth rate 

0.0332 

1,175 

1,481.65 

0.000 

Level-2 school initial status 

0.0430 

39 

77.82 

0.000 

Level-2 school growth rate 

0.0144 

39 

113.32 

0.000 

Learner-Centered Instruction 

Level- 1 temporal variation 

0.7106 




Level-2 individual initial status 

1.0114 

1,210 

3,142.50 

0.000 

Level-2 individual growth rate 

0.0274 

1,210 

1,444.65 

0.000 

Level-2 school initial status 

0.0240 

39 

74.97 

0.001 

Level-2 school growth rate 

0.0065 

39 

81.01 

0.000 

Resistance to Integration 

Level- 1 temporal variation 

0.8171 




Level-2 individual initial status 

0.7750 

1,208 

2,553.34 

0.000 

Level-2 individual growth rate 

0.0695 

1,208 

1,594.95 

0.000 

Level-2 school initial status 

0.0037 

39 

46.54 

0.190 

Level-2 school growth rate 

0.0009 

39 

49.42 

0.122 

Student Classroom Activities 

Level- 1 temporal variation 

0.2123 




Level-2 individual initial status 

0.2959 

1,213 

3,038.48 

0.000 

Level-2 individual growth rate 

0.0057 

1,213 

1,470.93 

0.000 

Level-2 school initial status 

0.0244 

39 

109.62 

0.000 

Level-2 school growth rate 

0.0039 

39 

111.86 

0.000 

Collaboration 

Level- 1 temporal variation 

0.2479 




Level-2 individual initial status 

0.2709 

1,219 

2,670.10 

0.000 

Level-2 individual growth rate 

0.0108 

1,219 

1,499.93 

0.000 

Level-2 school initial status 

0.0338 

39 

126.55 

0.000 

Level-2 school growth rate 

0.0048 

39 

110.16 

0.000 
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Researchers also used HLM growth models to estimate immersion effects on teacher mediating variables, 
controlling for teacher characteristics. Statistical details for these models are provided in Tables E.4, E.5 
and E.6. 


Table E.4. Descriptive Statistics for Teacher Variables: 
HLM models with Teacher Characteristics 


Variable Name 

N 

Mean 

SD 

Repeated Measures Descriptive Statistics (Level 1) 

Time 

6,736 

1.50 

1.12 

Technology Proficiency 

4,395 

4.87 

1.41 

Professional Productivity 

4,357 

3.23 

0.72 

Technology Integration 

4,146 

3.66 

1.56 

Learner-Centered Instruction 

4,289 

4.10 

1.37 

Resistance to Integration 

4,316 

2.32 

1.35 

Student Classroom Activities 

4,326 

2.14 

0.80 

Collaboration 

4,356 

2.51 

0.77 

Teacher-Level Descriptive Statistics (Level 2) 

Male 

1,684 

0.33 

0.47 

Hispanic 

1,684 

0.35 

0.48 

African American 

1,684 

0.05 

0.21 

Experience 

1,684 

12.28 

9.62 

School-Level Descriptive Statistics (Level 3) 

Immersion status (1 = yes, 0 = no) 

42 

0.50 

0.51 


Table E.5. Immersion (Fixed) Effect Analyses of Teacher-Level Variables: 
HLM Models with Teacher Characteristics 


School-Level Scale 

School-Level 

Analysis 

Gamma 

Coefficient 

Standard 

Error 

t 

Technology Proficiency 


Initial status (fall 2004) 

4.805 

0.083 

57.69*** 


Immersion dummy 3 

-0.201 

0.095 

-2.11* 


Male 

-0.076 

0.091 

-0.84 


Hispanic 

-0.183 

0.089 

-2.05* 


African American 

-0.016 

0.124 

-0.13 


Experience 

-0.053 

0.004 

-11 71*** 


Growth rate 

0.131 

0.022 

6.03*** 


Immersion dummy 3 

0.171 

0.032 

5.31*** 


Male 

-0.039 

0.027 

-1.44 


Hispanic 

0.005 

0.036 

0.13 


African American 

0.006 

0.038 

0.16 


Experience 

0.003 

0.001 

3.21** 


( Continued ) 
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Table E.5. Immersion (Fixed) Effect Analysis of Teacher-Level Variables (Continued) 


Professional Productivity 

Initial status (fall 2004) 

3.102 

0.060 

51.59*** 

Immersion dummy 

-0.095 

0.077 

-1.23 

Male 

-0.193 

0.056 

-3.42** 

Hispanic 

0.019 

0.042 

0.45 

African American 

0.090 

0.072 

1.25 

Experience 

-0.015 

0.003 

-5.02*** 

Growth rate 

0.073 

0.013 

5.59*** 

Immersion dummy 

0.116 

0.019 

6.26*** 

Male 

0.003 

0.018 

0.15 

Hispanic 

0.005 

0.014 

0.34 

African American 

0.030 

0.030 

0.97 

Experience 

0.002 

0.001 

1.88f p 

Technology Integration 

Initial status (fall 2004) 

2.869 

0.073 

39.44*** 

Immersion dummy b 

0.307 

0.102 

3.03** 

Male 

-0.188 

0.075 

-2.51* 

Hispanic 

0.304 

0.099 

3.09** 

African American 

0.562 

0.141 

3 99*** 

Experience 

-0.016 

0.004 

-3.69*** 

Growth rate 

0.256 

0.036 

7 \ 1 *** 

Immersion dummy b 

0.345 

0.051 

6.78*** 

Male 

-0.020 

0.031 

-0.64 

Hispanic 

-0.092 

0.039 

-2.38* 

African American 

0.048 

0.075 

0.64 

Experience 

0.002 

0.001 

2.04* 

Learner-Centered Instruction 

Initial status (fall 2004) 

3.701 

0.064 

58.13*** 

Immersion dummy 

-0.021 

0.082 

-0.26 

Male 

-0.229 

0.068 

-3.36** 

Hispanic 

0.227 

0.092 

2.47** 

African American 

0.532 

0.110 

4 32 *** 

Experience 

-0.025 

0.004 

-6.34*** 

Growth rate 

0.196 

0.028 


Immersion dummy 

0.184 

0.040 

4 57*** 

Male 

-0.011 

0.036 

-0.30 

Hispanic 

-0.034 

0.038 

-0.90 

African American 

0.025 

0.069 

0.36 

Experience 

0.004 

0.002 

2.14* 


(Continued) 
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Table E.5. Immersion (Fixed) Effect Analysis of Teacher-Level Variables (Continued) 


Resistance to Integration 

Initial status (fall 2004) 

2.395 

0.061 

39.25*** 

Immersion dummy 0 

-0.296 

0.058 

-5.11*** 

Male 

0.423 

0.077 

5.48*** 

Hispanic 

-0.303 

0.066 

-4.56*** 

African American 

-0.451 

0.084 

-5.38*** 

Experience 

0.012 

0.004 

3.05** 

Growth rate 

0.010 

0.024 

0.41 

Immersion dummy 0 

-0.015 

0.031 

-0.47 

Male 

0.024 

0.039 

0.62 

Hispanic 

0.058 

0.031 

1.88t 

African American 

0.075 

0.060 

1.24 

Experience 

-0.003 

0.002 

-1.37 

Student Classroom Activities 

Initial status (fall 2004) 

1.837 

0.047 

39.25*** 

Immersion dummy 

0.083 

0.058 

1.44 

Male 

-0.040 

0.062 

-0.65 

Hispanic 

0.201 

0.046 

4 38*** 

African American 

0.376 

0.089 

4 24*** 

Experience 

-0.005 

0.002 

-2.37* 

Growth rate 

0.036 

0.016 

2.32* 

Immersion dummy 

0.196 

0.026 

7.53*** 

Male 

0.012 

0.025 

0.49 

Hispanic 

-0.021 

0.018 

-1.15 

African American 

-0.031 

0.050 

-0.61 

Experience 

0.000 

0.001 

0.08 

Collaboration 

Initial status (fall 2004) 

2.273 

0.048 

47.42*** 

Immersion dummy 

0.113 

0.065 

1.74f 

Male 

-0.021 

0.049 

-0.43 

Hispanic 

0.135 

0.048 

2.82** 

African American 

0.354 

0.086 

4 i 3 *** 

Experience 

-0.007 

0.002 

-3.63** 

Growth rate 

0.030 

0.020 

1.54 

Immersion dummy 

0.065 

0.028 

2.30* 

Male 

0.004 

0.016 

0.28 

Hispanic 

0.000 

0.023 

-0.02 

African American 

-0.050 

0.050 

-1.00 

Experience 

0.001 

0.001 

1.28 


f p < .10; *p < .05; **p < .01; ***p < .001. 

“Immersed teachers had significantly lower initial technology proficiency scores. A latent variable regression, controlling for 
the effect of this initial difference on the growth rate, indicated that the difference between the original and adjusted 
immersion coefficients was not significant (the difference divided by the standard error of the difference = 1.05). 
b Immersed teachers had significantly lower technology integration scores. A latent variable regression, controlling for the 
effect of this initial difference on the growth rate, indicated that the difference between the original and adjusted immersion 
coefficients was not significant (the difference divided by the standard error of the difference = -0.77). 

“Immersed teachers had significantly higher initial levels of resistance to integration scores. A latent variable regression, 
controlling for the effect of this initial difference on the growth rate, indicated that the difference between the original and 
adjusted immersion coefficients was not significant (the difference divided by the standard error of the difference = -0.22). 
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Table E.6. Variance Decomposition from Conditional HLM Growth Models of Teacher 
Mediating Variables (with Teacher Characteristics) 


Scale/ 

Random Effect 

Variance 

Component 

df 

X 2 

P 

Technology Proficiency 

Level- 1 temporal variation 

0.3274 




Level-2 individual initial status 

1.5672 

1,223 

7,562.31 

0.000 

Level-2 individual growth rate 

0.0308 

1,223 

1,760.63 

0.000 

Level-2 school initial status 

0.0314 

40 

74.24 

0.001 

Level-2 school growth rate 

0.0049 

40 

95.01 

0.000 

Professional Productivity 

Level- 1 temporal variation 

0.1502 




Level-2 individual initial status 

0.2955 

1,217 

3,813.60 

0.000 

Level-2 individual growth rate 

0.0097 

1,217 

1,602.71 

0.000 

Level-2 school initial status 

0.0440 

40 

157.90 

0.000 

Level-2 school growth rate 

0.0013 

40 

74.89 

0.001 

Technology Integration 

Level- 1 temporal variation 

0.6960 




Level-2 individual initial status 

1.2380 

1,171 

3,394.33 

0.000 

Level-2 individual growth rate 

0.0341 

1,171 

1,481.38 

0.000 

Level-2 school initial status 

0.0369 

40 

71.07 

0.002 

Level-2 school growth rate 

0.0154 

40 

113.22 

0.000 

Learner-Centered Instruction 

Level- 1 temporal variation 

0.7095 




Level-2 individual initial status 

0.9432 

1,206 

3,014.05 

0.000 

Level-2 individual growth rate 

0.0266 

1,206 

1,436.20 

0.000 

Level-2 school initial status 

0.0136 

40 

64.10 

0.009 

Level-2 school growth rate 

0.0071 

40 

81.83 

0.000 

Resistance to Integration 

Level- 1 temporal variation 

0.8185 




Level-2 individual initial status 

0.6967 

1,204 

2,403.16 

0.000 

Level-2 individual growth rate 

0.0684 

1,204 

1,585.94 

0.000 

Level-2 school initial status 

0.0027 

40 

42.56 

0.361 

Level-2 school growth rate 

0.0011 

40 

49.54 

0.143 

Student Classroom Activities 

Level- 1 temporal variation 

0.2128 




Level-2 individual initial status 

0.2867 

1,209 

2,969.13 

0.000 

Level-2 individual growth rate 

0.0055 

1,209 

1,470.60 

0.000 

Level-2 school initial status 

0.0161 

40 

91.50 

0.000 

Level-2 school growth rate 

0.0039 

40 

113.24 

0.000 

Collaboration 

Level- 1 temporal variation 

0.2484 




Level-2 individual initial status 

0.2622 

1,215 

2,614.80 

0.000 

Level-2 individual growth rate 

0.0106 

1,215 

1,499.42 

0.000 

Level-2 school initial status 

0.0242 

40 

108.11 

0.000 

Level-2 school growth rate 

0.0046 

40 

107.76 

0.000 
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Effects of Technology Immersion on Students and Learning (Chapter 5) 

For the results reported in Chapter 5, researchers analyzed the effects of immersion on student mediating 
variables for Cohorts 1, 2, and 3 using two- and three-level FILM models. 

Effects on Mediating Variables 

Cohorts 1 and 2. In spring 2007, student surveys were not administered at three campuses, two treatment 
and one control school. For both Cohorts 1 and 2, we used AMOS 7.0 to perform model-based imputation 
to predict 2007 school technology (3 scales) and self-perception (3 scales) scale scores for the students at 
the three campuses with missing surveys. Our student-level model predicted the spring 2007 student scale 
score from the spring 2006 scale score, gender (1 if female, 0 if male), African-American status (1 if 
African American, 0 if not), Hispanic status (1 if Hispanic, 0 if not), economic status (1 if on free- or 
reduced-lunch, 0 if not), and immersion status (1 if the student attended an immersion campus, 0 if he or 
she attended a control campus). The result was five complete datasets for each scale for Cohorts 1 and 2. 

These multiply-imputed datasets were then analyzed using HLM 6.04. (Note that HLM results from 10 
imputed datasets were compared to the results from 5 imputed datasets, and there were essentially no 
differences in the coefficients. The reduced number of imputed datasets made the HLM analyses 
mechanically easier to run.) Specifically, researchers used three-level HLM growth models, with controls 
for school poverty (percentage of economically disadvantaged students) and student poverty 
(qualification for free- or reduced-price lunch). The models’ simplicity aids in the interpretation of 
effects. Statistical details are provided in Tables E.7, E.8, and E.9 for analyses of mediating variables for 
Cohort 1. We also analyzed more complex growth models, including controls for other student 
demographic variables (gender, ethnicity, home Internet access). Since these models yielded immersion 
growth coefficients nearly identical to those reported in Tables E.7, E.8 and E.9, results arc not provided 
in this report. 


Table E.7. Descriptive Statistics for Student Variables, Cohort 1 


Variable Name 

N 

Mean a 

SD 

Repeated Measure Descriptive Statistics (Level 1) 

Time 

11,705 

1.50 

1.12 

Time (SLI) 

10,880 

1.50 

1.12 

Technology Proficiency score 

11,124 

3.27 

0.89 

Technology Use in School score 

10,552 

2.32 

0.81 

Technical Problems score 

10,795 

2.50 

0.92 

Small-Group Work score 

10,643 

2.79 to 2.80 

0.86 

School Satisfaction score 

10,752 

3.68 

0.75 

Self-Directed Learning score 

10,396 

4.45 

0.74 

Student-Level Descriptive Statistics (Level 2) 

Eco. disadvantaged (1 = yes, 0 = no) 

2,942 

0.72 

0.45 

Eco. disadvantaged (SLI) 

2,720 

0.72 

0.45 

School-Level Descriptive Statistics (Level 3) 

Immersion status (1 = yes, 0 = no) 

42 

0.50 

0.51 

School poverty (proportion) 

42 

0.70 

0.17 


J Range of imputed means is listed when means differed across imputations. 
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Table E.8. Immersion (Fixed) Effect Analyses of Student Mediating Variables, Cohort 1 


School-Level Scale 

School-Level 

Analysis 

Gamma 

Coefficient 

Standard 

Error 

t 

Technology Proficiency 


Initial status (fall 2004) 

2.978 

0.060 

49.45*** 


Immersion dummy 

0.043 

0.077 

0.56 


School poverty 

-0.126 

0.231 

-0.54 


Disadvantaged 

-0.341 

0.039 

-8.75*** 


Growth rate 

0.267 

0.015 

18.19*** 


Immersion dummy 

0.043 

0.028 

1.53 


School poverty 

0.005 

0.083 

0.06 


Disadvantaged 

0.008 

0.013 

0.64 


Disadv. x Immersion 

0.058 

0.014 

4 22*** 

Self-Directed Learning 


Initial status (fall 2004) 

4.552 

0.051 

89.97*** 


Immersion dummy 3 

0.106 

0.050 

2.12* 


School poverty 

0.339 

0.153 

2.22* 


Disadvantaged 

-0.051 

0.046 

-1.11 


Growth rate 

-0.049 

0.014 

-3.60** 


Immersion dummy 3 

-0.017 

0.017 

-1.01 


School poverty 

-0.023 

0.042 

-0.55 


Disadvantaged 

-0.023 

0.014 

-1.68 

School Satisfaction 


Initial status (fall 2004) 

3.740 

0.030 

123.30*** 


Immersion dummy 

0.064 

0.038 

1.68 


School poverty 

-0.131 

0.121 

-1.08 


Disadvantaged 

-0.092 

0.033 

-2.79** 


Growth rate 

-0.055 

0.010 

_5.44*** 


Immersion dummy 

-0.008 

0.015 

-0.52 


School poverty 

0.163 

0.038 

4 33 *** 


Disadvantaged 

0.034 

0.012 

2.75** 

Classroom Activities (with technology) 


Initial status (fall 2004) 

2.020 

0.069 

29 37*** 


Immersion dummy b 

0.257 

0.079 

3.26** 


School poverty 

-0.298 

0.223 

-1.34 


Disadvantaged 

0.013 

0.030 

0.43 


Growth rate 

0.040 

0.025 

1.63 


Immersion dummy b 

0.212 

0.033 

6.34*** 


School poverty 

0.082 

0.096 

0.86 


Disadvantaged 

0.044 

0.014 

3.08** 

Small-Group Work 


Initial status (fall 2004) 

2.802 

0.058 

48.73*** 


Immersion dummy 

0.058 

0.058 

0.99 


School poverty 

-0.154 

0.194 

-0.80 


Disadvantaged 

-0.051 

0.053 

-0.96 


Growth rate 

-0.055 

0.021 

-2.56* 


Immersion dummy 

0.072 

0.026 

2.79** 


School poverty 

-0.052 

0.089 

-0.59 


Disadvantaged 

0.056 

0.020 

2.79** 


( Continued ) 
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Table E.8. Immersion (Fixed) Effect Analysis of Student Variables, Cohort 1 (Continued) 


Technical Problems 

Initial status (fall 2004) 

2.366 

0.063 

37.39*** 

Immersion dummy 

-0.138 

0.071 

-1.95 

School poverty 

-0.324 

0.249 

-1.30 

Disadvantaged 

-0.071 

0.069 

-1.04 

Growth rate 

0.101 

0.031 

3.27** 

Immersion dummy 

0.095 

0.039 

2.43* 

School poverty 

0.073 

0.137 

0.53 

Disadvantaged 

0.024 

0.026 

0.93 


*p < . 05 ; **p < . 01 ; ***p < . 001 . 

a Immersion students had significantly higher initial self-directed learning scores. A latent variable regression, controlling for the effect of this initial 
difference on the growth rate, indicated that the difference between the original and adjusted immersion coefficients was not significant (the difference 
divided by the standard error of the difference = 0.88). 

b Immersion students had significantly higher initial classroom activities scores. A latent variable regression, controlling for the effect of this initial 
difference on the growth rate, indicated that the difference between the original (0.138) and adjusted (0.212) immersion coefficients was significant (the 
difference divided by the standard error of the difference = -2.63). The growth rate coefficient adjusted for this difference is reported in the table. 


Table E.9. Variance Decomposition from Conditional HLM Models of Student Mediating Variables, 
Cohort 1 


Test/ 

Random Effect 

Variance 

Component 

df 

X 2 

P 

Technology Proficiency 

Level- 1 temporal variation 

0.2838 




Level-2 individual initial status 

0.4292 

2877 

8778.23 

0.000 

Level-2 individual growth rate 

0.0287 

2877 

4286.32 

0.000 

Level-3 school initial status 

0.0519 

39 

276.47 

0.000 

Level-3 school growth rate 

0.0061 

39 

265.29 

0.000 

Self-Directed Learning 

Level- 1 temporal variation 

0.2149 




Level-2 individual initial status 

0.3627 

2677 

8820.93 

0.000 

Level-2 individual growth rate 

0.0202 

2677 

3891.69 

0.000 

Level-3 school initial status 

0.0136 

39 

113.37 

0.000 

Level-3 school growth rate 

0.0015 

39 

96.58 

0.000 

School Satisfaction 

Level- 1 temporal variation 

0.3210 




Level-2 individual initial status 

0.2129 

2864 

5367.10 

0.000 

Level-2 individual growth rate 

0.0238 

2864 

3911.69 

0.000 

Level-3 school initial status 

0.0061 

39 

81.87 

0.000 

Level-3 school growth rate 

0.0007 

39 

65.70 

0.005 

Classroom Activities 

Level- 1 temporal variation 

0.4026 




Level-2 individual initial status 

0.1696 

2867 

4403.13 

0.000 

Level-2 individual growth rate 

0.0060 

2867 

3075.13 

0.004 

Level-3 school initial status 

0.0520 

39 

274.98 

0.000 

Level-3 school growth rate 

0.0115 

39 

321.26 

0.000 


(Continued) 
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Table E.9. Variance Decomposition from Conditional HLM Models of Student Mediating 
Variables, Cohort 1 (Continued) 


Small-Group Work 

Level- 1 temporal variation 

0.4857 




Level-2 individual initial status 

0.2520 

2864 

4807.28 

0.000 

Level-2 individual growth rate 

0.0268 

2864 

3640.40 

0.000 

Level-3 school initial status 

0.0235 

39 

132.99 

0.000 

Level-3 school growth rate 

0.0044 

39 

122.87 

0.000 

Technical Problems 

Level- 1 temporal variation 

0.5491 




Level-2 individual initial status 

0.2410 

2871 

4514.92 

0.000 

Level-2 individual growth rate 

0.0264 

2871 

3545.61 

0.000 

Level-3 school initial status 

0.0324 

39 

176.11 

0.000 

Level-3 school growth rate 

0.0108 

39 

233.23 

0.000 


Statistical details are provided in Tables E.10, E. 11, and E.12 for analyses of mediating variables for 
Cohort 2. Comparable to Cohort 1, analyses for more complex growth models, including controls for 
other student demographic variables (gender, ethnicity, home Internet access) yielded immersion growth 
coefficients nearly identical to those reported in Tables E. 10, E. 1 1 and E.12., so results are not provided 
in this report. 


Table E.10. Descriptive Statistics for Student Variables, Cohort 2 


Variable Name 

N 

Mean a 

SD 

Repeated Measure Descriptive Statistics (Level 1) 

Time 

10,153 

1.05 

0.86 

Time (SLI) 

8,148 

1.00 

0.82 

Technology Proficiency score 

9,638 

3.21 

0.90 

Technology Use in School score 

9,215 

2.34 

0.84 

Technical Problems score 

9,339 

2.34 

0.92 

Small-Group Work score 

9,238 

2.79 to 2.80 

0.88 

School Satisfaction score 

9,345 

3.71 to 3.72 

0.74 

Self-Directed Learning score 

8,053 

4.55 

0.74 

Student-Level Descriptive Statistics (Level 2) 

Eco. disadvantaged (1 = yes, 0 = no) 

3,397 

0.72 

0.45 

Eco. disadvantaged (SLI) 

2,716 

0.70 

0.46 

School-Level Descriptive Statistics (Level 3) 

Immersion status ( 1 = yes, 0 = no) 

42 

0.50 

0.51 

School poverty (proportion) 

42 

0.70 

0.17 


J Range of imputed means is listed when means differed across imputations. 
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Table E.11. Immersion (Fixed) Effect Analyses of Student Mediating Variables, Cohort 2 


School-Level Scale 

School-Level 

Analysis 

Gamma 

Coefficient 

Standard 

Error 

t 

Technology Proficiency 


Initial status (fall 2004) 

2.993 

0.061 

49.17*** 


Immersion dummy 

0.009 

0.077 

0.11 


School poverty 

0.215 

0.231 

0.93 


Disadvantaged 

-0.290 

0.043 

-6.75*** 


Growth rate 

0.268 

0.033 

g 23*** 


Immersion dummy 

0.160 

0.038 

4 25*** 


School poverty 

-0.149 

0.124 

-1.20 


Disadvantaged 

0.006 

0.032 

0.20 

Self-Directed Learning 


Initial status (fall 2004) 

4.787 

0.053 

90.41*** 


Immersion dummy 

-0.070 

0.064 

-1.10 


School poverty 

0.303 

0.187 

1.62 


Disadvantaged 

-0.129 

0.036 

-3.57** 


Growth rate 

-0.137 

0.019 

_7 \ j*** 


Immersion dummy 

0.024 

0.027 

0.87 


School poverty 

-0.026 

0.082 

-0.32 


Disadvantaged 

0.004 

0.018 

0.22 

School Satisfaction 


Initial status (fall 2004) 

3.865 

0.035 

111.65*** 


Immersion dummy 

0.034 

0.043 

0.79 


School poverty 

-0.063 

0.097 

-0.65 


Disadvantaged 

-0.138 

0.023 

-5.90*** 


Growth rate 

-0.075 

0.021 

-3.66** 


Immersion dummy 

-0.021 

0.027 

-0.78 


School poverty 

0.077 

0.057 

1.34 


Disadvantaged 

0.026 

0.017 

1.55 

Classroom Activities (with technology) 


Initial status (fall 2004) 

2.083 

0.064 

32.57*** 


Immersion dummy 

0.145 

0.093 

1.57 


School poverty 

0.454 

0.275 

1.65 


Disadvantaged 

-0.013 

0.039 

-0.34 


Growth rate 

0.059 

0.042 

1.40 


Immersion dummy 

0.244 

0.059 

4.16*** 


School poverty 

-0.174 

0.165 

-1.05 


Disadvantaged 

0.039 

0.024 

1.62 


( Continued ) 
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Table E.11. Immersion (Fixed) Effect Analysis of Student Variables, Cohort 2 
(Continued) 


Small-Group Work 


Initial status (fall 2004) 

2.786 

0.061 

45.50*** 


Immersion dummy 

-0.062 

0.081 

-0.76 


School poverty 

0.144 

0.177 

0.81 


Disadvantaged 

- 0.010 

0.042 

-0.25 


Growth rate 

- 0.011 

0.030 

-0.35 


Immersion dummy 

0.150 

0.041 

3.62** 


School poverty 

-0.023 

0.115 

- 0.20 


Disadvantaged 

-0.013 

0.022 

-0.61 

Technical Problems 


Initial status (fall 2004) 

2.176 

0.055 

39.28*** 


Immersion dummy“ 

-0.295 

0.075 

-3 94 *** 


School poverty 

0.264 

0.222 

1.19 


Disadvantaged 

-0.044 

0.028 

-1.56 


Growth rate 

0.168 

0.042 

3.99** 


Immersion dummy“ 

0.072 

0.048 

1.50 


School poverty 

-0.073 

0.172 

-0.42 


Disadvantaged 

0.004 

0.021 

0.17 


*p < .05; **p < .01; ***p < .001. 

“Immersion students had significantly lower initial technical problems scores. A latent variable regression, 
controlling for the effect of this initial difference on the growth rate, indicated that the difference between the 
original (0.254) and adjusted (0.072) immersion coefficients was significant (the difference divided by the 
standard error of the difference = 3.32). The growth rate coefficient adjusted for this difference is reported in the 
table. 


Table E.12. Variance Decomposition from Conditional HLM Models of Student Mediating Variables, 
Cohort 2 


Test/ 

Random Effect 

Variance 

Component 

df 

X 2 

P 

Technology Proficiency 

Level- 1 temporal variation 

0.2960 




Level-2 individual initial status 

0.4483 

3299 

8982.79 

0.000 

Level-2 individual growth rate 

0.0545 

3299 

4526.94 

0.000 

Level-3 school initial status 

0.0530 

39 

233.90 

0.000 

Level-3 school growth rate 

0.0110 

39 

190.34 

0.000 

Self-Directed Learning 

Level- 1 temporal variation 

0.2098 




Level-2 individual initial status 

0.3274 

2673 

7554.14 

0.000 

Level-2 individual growth rate 

0.0452 

2673 

3795.21 

0.000 

Level-3 school initial status 

0.0317 

39 

190.37 

0.000 

Level-3 school growth rate 

0.0040 

39 

112.78 

0.000 

School Satisfaction 

Level- 1 temporal variation 

0.3429 




Level-2 individual initial status 

0.1968 

3252 

5296.02 

0.000 

Level-2 individual growth rate 

0.0226 

3252 

3663.38 

0.000 

Level-3 school initial status 

0.0096 

39 

95.16 

0.000 

Level-3 school growth rate 

0.0037 

39 

97.97 

0.000 
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Table E.12. Variance Decomposition from Conditional HLM Models of Student Mediating Variables, 
Cohort 2 (Continued) 


Classroom Activities 

Level- 1 temporal variation 

0.4291 




Level-2 individual initial status 

0.1933 

3255 

4861.99 

0.000 

Level-2 individual growth rate 

0.0170 

3255 

3491.84 

0.002 

Level-3 school initial status 

0.0811 

39 

337.03 

0.000 

Level-3 school growth rate 

0.0334 

39 

404.17 

0.000 

Small-Group Work 

Level- 1 temporal variation 

0.5396 




Level-2 individual initial status 

0.2229 

3248 

4682.94 

0.000 

Level-2 individual growth rate 

0.0358 

3248 

3659.85 

0.000 

Level-3 school initial status 

0.0534 

39 

224.39 

0.000 

Level-3 school growth rate 

0.0111 

39 

145.32 

0.000 

Technical Problems 

Level- 1 temporal variation 

0.4986 




Level-2 individual initial status 

0.1920 

3258 

4666.88 

0.000 

Level-2 individual growth rate 

0.0605 

3258 

4032.74 

0.000 

Level-3 school initial status 

0.0433 

39 

214.26 

0.000 

Level-3 school growth rate 

0.0276 

39 

269.24 

0.000 


Cohort 3. Researchers’ used two-level HLM models, with controls for fall scale scores, student 
demographic characteristics (gender, ethnicity, economic disadvantage), and school poverty (percentage 
of economically disadvantaged students) and student poverty (qualification for free- or reduced-price 
lunch), to estimate the effects of immersion on student mediating variables for Cohort 3. Statistical details 
for are provided in Tables E. 13, E.14, and E. 15. 


Table E.13 Descriptive Statistics for Student Mediating Variables, Cohort 3 


Variable Name 

N 

Mean 

SD 

Student-Level Descriptive Statistics (Level 1) 

Female 

3,980 

0.51 

0.50 

Hispanic 

3,983 

0.70 

0.46 

African American 

3,983 

0.06 

0.23 

Eco. disadvantaged (1 = yes, 0 = no) 

3,969 

0.73 

0.44 

Technology Proficiency (fall 2006) 

3,893 

2.89 

0.90 

Technology Proficiency (spring 2007) 

3,927 

3.28 

0.85 

Self-Directed Learning (fall 2006) 

3,931 

4.72 

0.69 

Self-Directed Learning (spring 2007) 

3,904 

4.52 

0.71 

Classroom Activities (fall 2006) 

3,781 

2.16 

0.90 

Classroom Activities (spring 2007) 

3,892 

2.41 

0.84 

Technical Problems (fall 2006) 

3,813 

2.01 

0.87 

Technical Problems (spring 2007) 

3,871 

2.40 

0.90 

Small-Group Work (fall 2006) 

3,801 

2.79 

0.90 

Small-Group Work (spring 2007) 

3,835 

2.80 

0.87 

School Satisfaction (fall 2006) 

3,808 

3.77 

0.71 

School Satisfaction (spring 2007) 

3,840 

3.70 

0.76 

School-Level Descriptive Statistics (Level 2) 

Immersion status (1 = yes, 0 = no) 

39 

0.49 

0.51 

School poverty (proportion) 

39 

0.69 

0.17 
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Table E.14. Immersion (Fixed) Effect Analyses of Student Mediating Variables, Cohort 3 


School-Level Scale 

School-Level 

Analysis 

Gamma 

Coefficient 

Standard 

Error 

t 

Technology Proficiency 






Base 

3.144 

0.051 

62.11*** 


Immersion dummy 

0.330 

0.050 

6.56*** 


School poverty 

-0.345 

0.139 

-2.48* 


Female 

0.085 

0.028 

3.09** 


Hispanic 

-0.012 

0.032 

-0.37 


African American 

-0.024 

0.039 

-0.60 


Disadvantaged 

-0.073 

0.044 

-1.66 


Fall 2006 score 

0.526 

0.023 

23.36*** 

Self-Directed Learning 






Base 

4.468 

0.045 

99.06*** 


Immersion dummy 

0.056 

0.042 

1.32 


School poverty 

0.067 

0.131 

0.51 


Female 

0.073 

0.023 

3.22** 


Hispanic 

0.017 

0.045 

0.38 


African American 

0.094 

0.043 

2.16* 


Disadvantaged 

-0.036 

0.019 

-1.90 


Fall 2006 score 

0.571 

0.022 

25.39*** 

School Satisfaction 






Base 

3.625 

0.047 

77 67*** 


Immersion dummy 

0.068 

0.039 

1.75 


School poverty 

0.075 

0.157 

0.48 


Female 

0.120 

0.023 

5 27*** 


Hispanic 

0.026 

0.046 

0.57 


African American 

-0.017 

0.060 

-0.29 


Disadvantaged 

-0.038 

0.035 

-1.09 


Fall 2006 score 

0.396 

0.032 

12 49*** 

Classroom Activities 





(with technology) 

Base 

1.998 

0.056 

35.66*** 


Immersion dummy 

0.610 

0.072 

8.51*** 


School poverty 

-0.174 

0.228 

-0.76 


Female 

0.004 

0.023 

0.18 


Hispanic J 

0.046 

0.029 

1.57 


African American 

0.138 

0.058 

2.38* 


Disadvantaged 

0.112 

0.031 

3.66** 


Fall 2006 score 

0.257 

0.025 

10.28*** 

Small-Group Work 






Base 

2.542 

0.055 

46.54*** 


Immersion dummy 

0.305 

0.073 

4 19*** 


School poverty 

0.060 

0.210 

0.28 


Female 

0.076 

0.022 

3.50** 


Hispanic 

0.044 

0.034 

1.30 


African American 

0.207 

0.042 

4.88*** 


Disadvantaged 

0.059 

0.041 

1.44 


Fall 2006 score 

0.254 

0.016 

15.65*** 

Technical Problems 






Base 

2.243 

0.072 

31 22*** 


Immersion dummy 

0.209 

0.079 

2.65* 


School poverty 

0.058 

0.258 

0.23 


Female 

0.034 

0.029 

1.17 


Hispanic 

0.008 

0.052 

0.15 


African American 

0.007 

0.058 

0.11 


Disadvantaged 

-0.010 

0.036 

-0.29 


Fall 2006 score 

0.224 

0.026 

8.48*** 


*p < .05; **p < .01; ***p < .001. 
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Table E.15. Variance Decomposition from Conditional HLM Models of Student Mediating Variables, 
Cohort 3 


Test/ 

Variance 




Random Effect 

Component 

df 

X 2 

P 

Technology Proficiency 

Level- 1 student effect 

0.4520 




School mean 

0.0198 

36 

160.77 

0.000 

School pretest-outcome slope 

0.0109 

38 

90.61 

0.000 

Self-Directed Learning 

Level- 1 student effect 

0.3233 




School mean 

0.0179 

35 

163.53 

0.000 

School pretest-outcome slope 

0.0104 

37 

89.71 

0.000 

School Satisfaction 

Level- 1 student effect 

0.4779 




School mean 

0.0095 

36 

89.07 

0.000 

School pretest-outcome slope 

0.0214 

38 

102.57 

0.000 

Classroom Activities 

Level- 1 student effect 

0.4916 




School mean 

0.0468 

36 

228.26 

0.000 

School pretest-outcome slope 

0.0112 

38 

100.18 

0.000 

Small-Group Work 

Level- 1 student effect 

0.6542 




School mean 

0.0455 

36 

197.77 

0.000 

School pretest-outcome slope 

Effect not random 

Technical Problems 

Level- 1 student effect 

0.7030 




School mean 

0.0631 

36 

235.59 

0.000 

School pretest-outcome slope 

0.0122 

38 

71.58 

0.001 


Effects on School Attendance 

Comparable to analyses for student-level valuables, we used three-level HLM growth models and two- 
level HLM analyses to estimate the effects of immersion on student attendance. Statistical details are 
provided in Tables E.16, E.17, and E.18. 
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Table E.16. Descriptive Statistics for Student Attendance 


1 Variable Name 

N 

Mean 

SD 1 

Cohort 1 Repeated Measures Descriptive Statistics (Level 1) 

Year 

16,680 

1.50 

1.12 

Attendance 

16,577 

96.69 

4.08 

Cohort 2 Repeated Measures Descriptive Statistics (Level 1) 

Year 

14,277 

1.00 

0.82 

Attendance 

14,130 

96.64 

4.11 

Cohort 1 Student-Level Descriptive Statistics (Level 2) 

Eco. disadvantaged (1 = yes, 0 = no) 

4,170 

0.74 

0.44 | 

Cohort 2 Student-Level Descriptive Statistics (Level 2) 

Eco. disadvantaged (1 = yes, 0 = no) 

4,759 

0.75 

0.43 I 

Cohort 3 Student-Level Descriptive Statistics (Level 1) 

Attendance 2007 

5,205 

96.41 

4.14 

Attendance 2006 

5,078 

97.03 

3.52 

Eco. disadvantaged (1 = yes, 0 = no) 

5,205 

0.76 

0.42 

Hispanic (1 = yes, 0 = no) 

5,205 

0.73 

0.44 

African American (1 = yes, 0 = no) 

5,205 

0.06 

0.23 

Female (1 = yes, 0 = no) 

5,205 

0.49 

0.50 

Cohorts 1, 2, and 3 School-Level Descriptive Statistics i 

Immersion status (1 = yes, 0 = no) 

42 

0.50 

0.51 

School poverty (proportion) 

42 

0.70 

0.17 


Table E.17. Immersion (Fixed) Effect Analyses of Student Attendance 


Group 

School-Level 

Analysis 

Gamma 

Coefficient 

Standard 

Error 

t 

Cohort 1 





3-Level HLM Growth Model 

Initial attendance (2004) 

97.706 

0.172 

567.25*** 


Immersion dummy 

-0.045 

0.242 

-0.19 


School poverty 

2.230 

0.627 

3.56** 


Eco. disadvantaged 

-0.651 

0.172 

-3 79 *** 


Growth rate 

-0.165 

0.061 

-2.72* 


Immersion dummy 

-0.272 

0.090 

-3.04** 


School poverty 

-0.260 

0.249 

-1.05 


Eco. disadvantaged 

-0.119 

0.059 

-2.03* 

Cohort 2 





3-Level HLM Growth Model 

Initial attendance (2004) 

97.518 

0.145 

671.67*** 


Immersion dummy 

-0.262 

0.185 

-1.41 


School poverty 

1.550 

0.572 

2.71* 


Eco. disadvantaged 

-0.487 

0.136 

-3.59** 


Growth rate 

- 0.120 

0.087 

-1.38 


Immersion dummy 

-0.249 

0.109 

-2.28* 


School poverty 

-0.437 

0.276 

-1.59 


Eco. disadvantaged 

-0.309 

0.073 

-4.26*** 

Cohort 3 





2-Level HLM Model 

Base 

96.886 

0.201 

482.35*** 


Immersion dummy 

-0.564 

0.211 

-2.67* 


School poverty 

0.393 

0.598 

0.66 


Female 

0.204 

0.104 

1.96* 


African American 

0.382 

0.242 

1.58 


Hispanic 

0.262 

0.172 

1.53 


Eco. disadvantaged 

-0.543 

0.115 

-4.74*** 


Prior attendance 

0.622 

0.037 

17.00*** 


i p < .10; *p < .05; **p < .01; ***p < .001. 
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Table E.18. Variance Decomposition from Conditional HLM Models of Student attendance, 
Cohorts 1, 2, and 3 


Cohort/ 
Random Effect 

Variance 

Component 

df 

V 2 

Pt 

Cohort 1 

Level- 1 temporal variation 

4.8370 


Level-2 individual initial status 

6.1984 

4127 

11612.89 

0.000 

Level-2 individual growth rate 

1.1460 

4127 

8997.61 

0.000 

Level-3 school initial status 

0.4884 

39 

227.83 

0.000 

Level-3 school growth rate 

0.0546 

39 

136.99 

0.000 

Cohort 2 

Level- 1 temporal variation 

4.9373 


Level-2 individual initial status 

7.3931 

4716 

13066.95 

0.000 

Level-2 individual growth rate 

1.6743 

4716 

7924.01 

0.000 

Level-3 school initial status 

0.1766 

39 

105.74 

0.000 

Level-3 school growth rate 

0.0670 

39 

100.96 

0.000 

Cohort 3 

Level- 1 student effect 

10.6080 


School mean 

0.5631 

39 

222.34 

0.000 

School pre-measure -outcome slope 

0.0410 

41 

322.08 

0.000 


Effects of Technology Immersion on Student Achievement (Chapter 6) 

Researchers used two-level HLM models and three-level HLM growth models to estimate the effects 
of immersion on student academic achievement. Statistical details are provided for Cohort 1 students 
(eighth graders) in Tables E. 19 through E.22, for Cohort 2 (seventh graders) in Tables E.23 through 
E.26, and for Cohort 3 students (sixth graders) in Tables E.27 through E.29. 

Cohort 1 (Eighth Graders) 


Table E.19. Descriptive Statistics for TAKS Reading and Mathematics, Cohort 1 


Variable Name 

N 

Mean 

SD 

Repeated Measures Descriptive Statistics: F 

leading (Level 1) 

Time 

11,972 

1.50 

1.12 

TAKS Reading T score 

11,972 

48.93 

9.40 

Repeated Measures Descriptive Statistics: Mathematics (Level 1) 

Time 

12,052 

1.50 

1.12 

TAKS Mathematics T score 

12,052 

49.15 

9.48 

Student-Level Descriptive Statistics: Reading (Level 2) 

Eco. disadvantaged (1 = yes, 0 = no) 

2,993 

0.70 

0.46 

Student-Level Descriptive Statistics: Mathematics (Level 2) 

Eco. disadvantaged (1 = yes, 0 = no) 

3,013 

0.70 

0.46 

School-Level Descriptive Statistics (Level 3) 


Immersion status ( 1 = yes, 0 = no) 

42 

0.50 

0.51 

School poverty (proportion) 

42 

0.70 

0.17 
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Table E.20. Descriptive Statistics for TAKS Science and Social Studies, Cohort 1 


Variable Name 

N 

Mean 

SD 

Student-Level Descriptive Statistics: Science (Level 1) 

Female 

2,967 

0.53 

0.50 

African American 

2,967 

0.05 

0.23 

Hispanic 

2,967 

0.69 

0.46 

Eco. disadvantaged (1 = yes, 0 = no) 

2,967 

0.69 

0.46 

TAKS Science T score (2004) 

2,967 

49.51 

9.17 

TAKS Science T score (2007) 

2,967 

48.93 

9.00 

Student-Level Descriptive Statistics: Social 

Studies (Level 1) 

Female 

3,143 

0.52 

0.50 

African American 

3,143 

0.05 

0.23 

Hispanic 

3,143 

0.69 

0.46 

Eco. disadvantaged (1 = yes, 0 = no) 

3,143 

0.70 

0.46 

TAKS Reading T score (2006) 

3,143 

48.77 

9.77 

TAKS Social Studies T score (2007) 

3,143 

48.86 

8.23 

School-Level Descriptive Statistics (Level 2) 


Immersion status (1 = yes, 0 = no) 

42 

0.50 

0.51 

School poverty (percent) 

42 

0.70 

0.17 


Table E.21. Immersion (Fixed) Effect Analyses of TAKS Achievement, Cohort 1 


TAKS Achievement Test 

School-Level 

Analysis 

Gamma 

Coefficient 

Standard 

Error 

/-value 

Reading 


Initial status (spring 2004) 

54.002 

0.702 

76.89*** 


Immersion dummy 3 

-1.346 

0.713 

-1.89f 


School poverty 

-6.504 

1.449 

_449*** 


Eco. disadvantaged 

-6.170 

0.657 

-9.40*** 


Growth rate 

-0.369 

0.129 

- 2 . 86 ** 


Immersion dummy 3 

0.212 

0.146 

1.45 


School poverty 

0.860 

0.493 

1.75 1 


Eco. disadvantaged 

0.536 

0.132 

4 Q7*** 

Mathematics 


Initial status (spring 2004) 

53.019 

0.749 

70.83*** 


Immersion dummy 

- 1.201 

0.882 

-1.36 


School poverty 

-4.724 

1.888 

-2.50* 


Eco. disadvantaged 

-4.487 

0.521 

-8.60*** 


Growth rate 

-0.181 

0.157 

-1.15 


Immersion dummy 

0.582 

0.298 

1.95 1 


School poverty 

1.488 

0.892 

1.67 


Eco. disadvantaged 

0.025 

0.086 

0.29 


Immersion dummy 

-0.408 

0.215 

-1.90 1 

Science 


Base 

52.269 

0.569 

91.89*** 


Immersion dummy 

-0.238 

0.668 

-0.36 


School poverty 

0.167 

1.830 

0.09 


Female 

-1.203 

0.204 

-5.90*** 


African American 

-2.052 

0.797 

-2.57* 


Hispanic 

-1.712 

0.383 

-4 47 *** 


Eco. disadvantaged 

-1.518 

0.344 

-4.41*** 


Spring 2004 T score 

0.681 

0.029 

23.72*** 


Immersion dummy 

-0.084 

0.044 

-1.90f 


( Continued ) 
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Table E.21. Immersion (Fixed) Effect Analyses of TAKS Achievement, Cohort 1 (Continued) 


1 Social Studies | 


Base 

49.525 

1.078 

45.93*** 


Immersion dummy 

0.574 

0.603 

0.95 


School poverty 

4.607 

1.658 

2.78** 


Female 

-2.537 

0.246 

-10.32*** 


African American 

-1.851 

0.349 

-5.31*** 


Hispanic 

-2.033 

0.510 

_3 99 *** 


Eco. disadvantaged 

-1.614 

0.371 

-4.35*** 


Spr. 2006 reading T score 

0.539 

0.015 

35.30*** 


< .10; *p < .05; **p < .01; ***p < .001. 

d Immersion students had significantly lower initial TAKS reading scores. A latent variable regression, controlling for the 
effect of this initial difference on the growth rate, indicated that the difference between the original and adjusted immersion 
coefficients was not significant (the difference divided by the standard error of the difference =1.30). 


Table E.22. Variance Decomposition from Conditional HLM Models of Student Achievement, 
Cohort 1 


Test/ 

Random Effect 

Variance 

Component 

df 

X 2 

P 

Reading 

Level- 1 temporal variation 

23.4384 




Level-2 individual initial status 

57.6673 

2950 

13315.15 

0.000 

Level-2 individual growth rate 

0.2738 

2950 

3117.01 

0.016 

Level-3 school initial status 

3.6305 

39 

171.19 

0.000 

Level-3 school growth rate 

0.1255 

39 

111.71 

0.000 

Mathematics 

Level- 1 temporal variation 

20.1761 




Level-2 individual initial status 

58.2327 

2970 

15210.61 

0.000 

Level-2 individual growth rate 

0.7539 

2970 

3525.93 

0.000 

Level-3 school initial status 

6.3585 

39 

230.66 

0.000 

Level-3 school growth rate 

0.7278 

39 

351.79 

0.000 

Science 

Level- 1 student effect 

34.0990 




School mean 

4.1842 

39 

213.97 

0.000 

School pre-measure -outcome 
slope 

0.0116 

40 

94.70 

0.000 

Social Studies 

Level- 1 student effect 

30.8751 




School mean 

3.8495 

39 

280.04 

0.000 

School pre-measure -outcome 
slope 

0.0047 

41 

75.56 

0.001 
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Cohort 2 (Seventh Graders) 

Table E.23 Descriptive Statistics for TAKS Reading and Mathematics, Cohort 2 


Variable Name 

N 

Mean 

SD 

Repeated Measures Descriptive Statistics: Reading (Level 1) 

Time 

9,813 

1.00 

0.82 

TAKS Reading T score 

9,813 

48.73 

9.48 

Repeated Measures Descriptive Statistics: Mathematics (Level 1) 

Time 

9,930 

1.00 

0.82 

TAKS Mathematics T score 

9,930 

48.53 

9.30 

Student-Level Descriptive Statistics: Reading (Level 2) 

Eco. disadvantaged (1 = yes, 0 = no) 

3,271 

0.70 

0.46 

Student-Level Descriptive Statistics 

: Mathematics (Level 2) 


Eco. disadvantaged (1 = yes, 0 = no) 

3,310 

0.70 

0.46 

School-Level Descriptive Statistics (Level 3) 

Immersion status (1 = yes, 0 = no) 

42 

0.50 

0.51 

School poverty (proportion) 

42 

0.70 

0.17 

Table E.24. Descriptive Statistics for TAKS Writing, Cohort 2 

Variable Name 

N 

Mean 

SD 

Student-Level Descriptive Statistics 

: Writing (Level 1) 


Female 

3,088 

0.52 

0.50 

African American 

3,088 

0.07 

0.26 

Hispanic 

3,088 

0.68 

0.47 

Eco. disadvantaged (1 = yes, 0 = no) 

3,088 

0.69 

0.46 

TAKS Writing T score (2004) 

3,088 

49.89 

9.39 

TAKS Writing T score (2007) 

3,088 

49.88 

7.53 

School-Level Descriptive Statistics (Level 2) 

Immersion status ( 1 = yes, 0 = no) 

42 

0.50 

0.51 

School poverty (percent) 

42 

0.70 

0.17 
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Table E.25. Immersion (Fixed) Effect Analyses of TAKS Achievement, Cohort 2 


TAKS Achievement Test 

School-Level 

Analysis 

Gamma 

Coefficient 

Standard 

Error 

f- value 

Reading I 


Initial status (spring 2005) 

52.770 

0.523 

100.87*** 


Immersion dummy 

-0.488 

0.601 

-0.81 


School poverty 

-8.049 

1.396 

-5 77 *** 


Eco. disadvantaged 

-5.511 

0.591 

-9 33*** 


Growth rate 

-0.155 

0.182 

-0.85 


Immersion dummy 

0.388 

0.233 

1.66 


School poverty 

0.905 

0.694 

1.30 


Eco. disadvantaged 

0.283 

0.182 

1.56 

Mathematics 1 


Initial status (spring 2005) 

52.306 

0.547 

95.59*** 


Immersion dummy 

-1.029 

0.667 

-1.54 


School poverty 

-4.373 

1.832 

-2.39* 


Eco. disadvantaged 

-4.492 

0.576 

-7 80*** 


Growth rate 

-0.444 

0.270 

-1.65 


Immersion dummy 

0.708 

0.398 

1.78f 


School poverty 

0.317 

1.022 

0.31 


Eco. disadvantaged 

0.044 

0.180 

0.24 

Writing 


Base 

51.181 

0.510 

100.44*** 


Immersion dummy 

-0.283 

0.428 

- 0.66 


School poverty 

0.315 

1.323 

0.24 


Female 

1.409 

0.237 

5 94 *** 


African American 

-1.141 

0.387 

-2.95** 


Hispanic 

-1.355 

0.303 

-4 47 *** 


Eco. disadvantaged 

- 1.210 

0.360 

-3.36** 


Spring 2004 T score 

0.453 

0.016 

28.59*** 


V < .10; *p < .05; **p < .01; ***p < .001. 


Table E.26. Variance Decomposition from Conditional HLM Models of Student Achievement, 
Cohort 2 


Test/ 

Random Effect 

Variance 

Component 

df 

X 2 

P 

Reading 

Level- 1 temporal variation 

26.0638 




Level-2 individual initial status 

50.0343 

3228 

21858.54 

0.000 

Level-2 individual growth rate 


Effect not random 


Level-3 school initial status 

1.9434 

39 

145.61 

0.000 

Level-3 school growth rate 

0.3064 

39 

111.80 

0.000 

Mathematics 

Level- 1 temporal variation 

19.5682 




Level-2 individual initial status 

54.9601 

3267 

14297.70 

0.000 

Level-2 individual growth rate 

1.5537 

3267 

3786.90 

0.000 

Level-3 school initial status 

3.0190 

39 

190.90 

0.000 

Level-3 school growth rate 

1.4050 

39 

481.48 

0.000 

Writing 

Level- 1 student effect 

31.3854 




School mean 

1.6384 

38 

151.64 

0.000 

School pre-measure -outcome slope 

0.0043 

40 

71.98 

0.002 
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Cohort 3 (Sixth Graders) 


Table E.27. Descriptive Statistics for TAKS Achievement, Cohort 3 


Variable Name 

N 

Mean 

SD 

Student-Level Descriptive Statistics: Reading (Level 1) 

Female 

3,769 

0.52 

0.50 

African American 

3,769 

0.06 

0.24 

Hispanic 

3,769 

0.70 

0.46 

Eco. disadvantaged (1 = yes, 0 = no) 

3,753 

0.72 

0.45 

TAKS Reading T score (2006) 

3,769 

49.28 

9.62 

TAKS Reading T score (2007) 

3,769 

48.62 

7.74 

Student-Level Descriptive Statistics : Mathematics (Level 1) 

Female 

3,813 

0.51 

0.50 

African American 

3,813 

0.06 

0.24 

Hispanic 

3,813 

0.70 

0.46 

Eco. disadvantaged (1 = yes, 0 = no) 

3,797 

0.72 

0.45 

TAKS Mathematics T score (2006) 

3,813 

49.44 

9.48 

TAKS Mathematics T score (2007) 

3,813 

48.51 

8.72 

School-Level Descriptive Statistics (Level 2) 

Immersion status (1 = yes, 0 = no) 

42 

0.50 

0.51 

School poverty (proportion) 

42 

0.70 

0.17 


Table E.28. Immersion (Fixed) Effect Analyses of Cohort 3 Achievement 


TAKS Achievement Test 

School-Level 

Analysis 

Gamma 

Coefficient 

Standard 

Error 

f-value 

Reading 


Base 

49.140 

0.337 

145.71*** 


Immersion dummy 

0.082 

0.249 

0.33 


School poverty 

1.591 

0.874 

1.82f 


Female 

1.385 

0.187 

7 4j*** 


African American 

-1.486 

0.352 

-4 23*** 


Hispanic 

-1.075 

0.277 

-3.88** 


Eco. disadvantaged 

-0.539 

0.192 

-2.81** 


Spring 2006 T score 

0.523 

0.015 

34 72*** 

Mathematics 


Base 

48.799 

0.491 

99.43*** 


Immersion dummy 

0.282 

0.640 

0.44 


School poverty 

0.265 

1.661 

0.16 


Female 

0.944 

0.233 

4.05*** 


African American 

-1.309 

0.355 

-3.69** 


Hispanic 

-0.486 

0.244 

-1.99* 


Eco. disadvantaged 

-1.064 

0.252 

-4 23*** 


Spring 2006 T score 

0.608 

0.021 

28.98*** 


Immersion dummy 

0.082 

0.024 

3.41** 


+ p < .10; *p < .05; **p < .01; ***p < .001. 
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Table E.29. Variance Decomposition from Conditional HLM Models of Student Achievement, 
Cohort 3 


Test/ 

Random Effect 

Variance 

Component 

df 

X 2 

P 

Reading 

Level- 1 student effect 

28.2895 




School mean 

1.1674 

39 

188.84 

0.000 

School pre-measure -outcome slope 

0.0045 

41 

99.62 

0.000 

Mathematics 

Level- 1 student effect 

33.7924 




School mean 

4.1607 

39 

469.85 

0.000 

School pre-measure -outcome slope 

0.0018 

40 

57.07 

0.039 
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Appendix F 

Effects of Technology Immersion on Schools 


On one part of the surveys, teachers responded to items pertaining to their perceptions of school-level 
supports for technology. Teachers were asked to rate the strength of their agreement with statements on a 
5-point scale ranging from 1 ( strongly disagree ) to 5 ( strongly agree). Four distinct organizational factors 
emerged from a factor analysis: Leadership (12 items), Innovative Culture (4 items), Parent and 
Community Support (2 items), and Technical Support (5 items). Cronbach’s alpha reliability coefficients 
for the scale scores ranged from acceptable (0.66) to excellent (0.97). 

In addition to using school-level supports for technology to measure the implementation levels of the 
technology immersion model, we also have used school-level scores to compare treatment and control 
teachers prior to technology immersion and after the first, second, and third implementation years. We 
analyzed the effects of immersion on teachers’ perceptions of school-level indicators of technology 
support using separate t tests to estimate differences between immersion and control teachers. We also 
calculated effect sizes (Cohen’s d) as a way to show the relative strength of differences. Effect size (ES) 
are expressed as standard deviations. For example, an effect size of 1.00 indicates teachers’ average score 
at an immersion school is one full standard deviation above the scores for control group teachers. Effect 
sizes can be interpreted generally as large (greater than 0.50), moderate (0.50-0.30), small (0.30-0.10), or 
trivial (less than 0.10). 

Effects on School-Level Technology Supports 

In Chapter 3, we reported that the strength of school-level supports for technology immersion varied 
across the 21 treatment campuses. Collectively, however, teachers at immersion schools perceived 
stronger technology supports than teachers at control campuses. Data from analyses of school variables, 
summarized in Table F.l, show that at the end of both the first implementation year (spring 2005) and 
second implementation year (spring 2006), teachers at immersion schools reported significantly higher 
scale scores than control teachers on items measuring their levels of agreement with the provision of four 
school-level supports for technology. Flowever, at the end of the third year of implementation (spring 
2007), there were no significant differences between immersion and control teachers on Leadership and 
Technical Support, and smaller significant differences on Teacher Support and Parent and Community 
Support. 

• Leadership (Effect sizes = 0.42 and 0.35, respectively, in spring 2005 and 2006, but only 0.03 in 
spring 2007), 

• Teacher Support (Effect sizes = 0.43, 0.40, and 0.27, respectively, in spring 2005, 2006, and 
2007), 

• Parent and Community Support (Effect sizes = 0.51, 0.24, and 0.16, respectively, in spring 2005, 
2006, and 2007), and 

• Technical Support (Effect sizes = 0.33, 0.20, and 0.07, respectively, in spring 2005, 2006, and 
2007). 

Teachers’ responses across time help to explain the effects of immersion on their perceptions of 
technology-related supports. 
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Leadership 


The scale score for Leadership indicates the extent to which teachers believe their administrators establish 
a clear vision and expectations for technology, encourage classroom integration, provide needed supports, 
and involve staff in decision making. Results show that in the fall of 2004, teachers at immersion schools 
reported significantly higher Leadership scores for their administrators than control teachers (M= 3.70 vs. 
3.60, ES = 0.16). This may have reflected teachers’ initial optimism about their administrators’ proactive 
efforts to secure Technology Immersion Pilot (TIP) grants. In the spring of 2005, after one year of project 
implementation, immersion teachers reported even stronger Leadership (ES = 0.42), and their higher 
estimations of administrative leadership, compared to control teachers, was sustained at the end of the 
second year (ES = 0.35). However, at the end of the third year, more teachers at control schools tended to 
agree that their administrators provided technology-related support, and there were essentially no 
differences between the two groups (M = 3.90 versus 3.88, ES = 0.03). This improvement in control 
teachers’ perceptions of technology-related leadership may reflect the emphasis on technology at their 
campuses. Most control campuses began planning to implement technology immersion during the next 
school year in at least one grade level. As paid of preparations for immersion, many control teachers 
received laptop computers, electronic resources, and technology-related professional development during 
the 2006-07 school year. 


Table F.l. Comparison Group Differences for School-Level Technology Support Variables 


School Variables 

Immersion 
N= 21 

Control 
N= 21 

f-value 

P 

Effect 

Size 

Mean 

SD 

Mean 

SD 

Leadership 

Fall 2004 

3.70 

0.61 

3.60 

0.67 

2.84 

0.005* 

0.16 

Spring 2005 

3.88 

0.62 

3.61 

0.67 

6.74 

0.000* 

0.42 

Spring 2006 

3.89 

0.65 

3.64 

0.76 

5.78 

0.000* 

0.35 

Spring 2007 

3.90 

0.73 

3.88 

0.68 

0.64 

0.521 

0.03 

Teacher Support (for innovation) 

Fall 2004 

3.72 

0.66 

3.71 

0.67 

0.23 

0.817 

0.02 

Spring 2005 

4.00 

0.60 

3.73 

0.65 

6.94 

0.000* 

0.43 

Spring 2006 

4.06 

0.62 

3.80 

0.68 

6.54 

0.000* 

0.40 

Spring 2007 

4.09 

0.63 

3.92 

0.68 

4.46 

0.000* 

0.26 

Parent and Community Support 

Fall 2004 

3.44 

0.78 

3.39 

0.74 

1.10 

0.273 

0.07 

Spring 2005 

3.75 

0.71 

3.38 

0.74 

8.05 

0.000* 

0.51 

Spring 2006 

3.64 

0.82 

3.44 

0.85 

3.89 

0.000* 

0.24 

Spring 2007 

3.69 

0.88 

3.56 

0.79 

2.86 

0.004* 

0.16 

Technical Support 

Fall 2004 

3.29 

0.75 

3.31 

0.76 

-0.45 

0.654 

-0.03 

Spring 2005 

3.60 

0.69 

3.36 

0.76 

5.12 

0.000* 

0.33 

Spring 2006 

3.63 

0.67 

3.49 

0.70 

3.34 

0.001* 

0.20 

Spring 2007 

3.70 

0.71 

3.65 

0.71 

1.06 

0.289 

0.07 


Notes. Scale scores range from 1.00 to 5.00. Fall 2004: N=524 to 533 immersion teachers, 1V=601 to 606 control teachers; 
spring 2005: A=468 to 481 immersion teachers and 1V=521 to 541 control teachers; spring 2006: 1V=507 to 519 immersion 
teachers and 1V=574 to 584 control teachers; spring 2007: N = 570 to 584 immersion teachers and 77=610 to 624 control 


teachers. *Statistically significant difference. Effect size is Cohen’s d. The effect size is interpreted as follows: a value greater 
than 0.5 is large, 0.5-0.3 is moderate, 0.3-0. 1 is small, and anything smaller than 0.1 is trivial. Port Arthur campuses were 
excluded. 
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Innovative Culture 

The scale score for Innovative Culture reflects the extent to which teachers at a school share an 
understanding about technology use, continually learn, are unafraid of new technologies, and arc 
generally supportive of technology integration efforts. In fall 2004, there were no significant differences 
between groups in teachers’ opinions. However, at the end of each implementation year, teachers at 
immersion schools were significantly more likely than control teachers to view their school culture as 
innovative (ES = 0.44, 0.40, and 0.26, respectively, in 2005, 2006, and 2007). Immersion teachers 
generally agreed with statements reflecting an Innovative Culture for technology. 

Parent and Community Support 

Scores for Parent and Community Support show the extent to which teachers believe that parents and the 
surrounding community support the school’s efforts with technology. There were no significant 
differences between immersion and control teachers’ views on the degree of support from parents and 
community members in fall of 2004, but at the end of the first implementation year (spring 2005), 
teachers at immersion schools reported a significantly stronger level of support than control teachers (ES 
= 0.50). After the second implementation year (spring 2006), immersion teachers’ positive perceptions of 
Parent and Community Support had waned, but their estimations of support still significantly exceeded 
control teachers’ views (ES = 0.24). Third year data show even smaller significant differences between 
immersion and control teachers (ES = 0.16). During the third year, control teachers’ scores increased 
more than did immersion teachers’ scores. This may also be due to many control campuses being in the 
planning phase for technology immersion. 

Technical Support 

The Technical Support scale indicates the extent to which teachers believe technical problems with 
computers, Internet access, repairs, and material availability pose banders to technology integration. In 
fall 2004, there were no significant differences between immersion and con tied teachers’ perceived 
support. However, with the infusion of technology resources and additional support staff through 
Technology Immersion Pilot (TIP) grants, immersion teachers reported significantly higher levels of 
Technical Support than control teachers at the end of the first and second implementation years (ES’s = 
0.33 and 0.22, respectively, in 2005 and 2006). Yet with control schools ramping up their technology 
preparation in the third year, there were no significant differences between immersion and control 
teachers’ perceived technical support (M = 3.70 immersion versus 3.65 control, ES = 0.07). 

In sum, after the first and second years of the pilot project, teachers at immersion schools compared to 
control perceived stronger school-level organizational supports for technology from administrators, 
parents and the community, and technical staff. And, as a whole, they also expressed greater affinity for 
innovative technology practices. Notably, however, teachers at control schools reported increasingly 
higher levels of agreement for each of the organizational support indicators after the third project year. 
This reflects the immersion planning process going on at most control campuses in preparation for giving 
laptops to students in at least one grade level during the next school year. 

An additional, longitudinal analysis for teachers who were in the immersion project from fall 2004 
through spring 2007 showed significant increases in immersion teachers’ reported agreement with the 
extent of support for each of the four indicators (similar to the cross-sectional findings). Longitudinal 
results for control teachers who were at the schools during the same time period also revealed statistically 
significant increases for all of the school-level indicators (see Table F.2). 
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Table F.2. School Variables for Immersed and Control Teachers in Fall 2004 and Spring 2007 


School Variables 

N 

Fall 2004 

Spring 2007 


P 

Mean 

SD 

Mean 

SD 

Lvalue 

Immersion 

Leadership and System Support 

314 

3.73 

0.60 

3.90 

0.74 

3.78 

0.000* 

Technical Support 

306 

3.25 

0.72 

3.68 

0.73 

8.86 

0.000* 

Innovative Culture 

305 

3.76 

0.62 

4.11 

0.62 

7.77 

0.000* 

Parent and Community Support 

314 

3.42 

0.78 

3.69 

0.90 

5.14 

0.000* 

Control 

Leadership and System Support 

375 

3.60 

0.68 

3.82 

0.72 

5.73 

0.000* 

Technical Support 

366 

3.31 

0.75 

3.66 

0.72 

8.27 

0.000* 

Innovative Culture 

367 

3.71 

0.66 

3.90 

0.68 

5.07 

0.000* 

Parent and Community Support 

371 

3.42 

0.73 

3.54 

0.84 

3.04 

0.003* 


Notes. Score range is from 1.00 to 5.00. Port Arthur campuses were excluded. 
*Statistically significant difference. 
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